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1.0 


EXECUTIVE  SUMMARY 


1.1  BACKGROUND 

Buried  unexploded  ordnance  (UXO)  is  arguably  the  most  serious  and  prevalent  environmental 
problem  currently  facing  Department  of  Defense  (DoD)  facility  managers.  Not  limited  to  active 
military  bases  and  test  ranges,  these  problems  also  occur  at  DoD  sites  that  are  currently  dormant 
and  in  areas  adjacent  to  military  ranges  that  belong  to  the  civilian  sector  or  are  under  control  of 
other  government  agencies.  The  amount  of  land  affected  is  generally  agreed  to  be  in  excess  of 
10  million  acres  in  the  continental  United  States.  UXO  mitigation  and  remediation  requirements 
assume  even  more  compelling  proportions  when  the  DoD  lands  involve  formerly  used  defense 
sites  (FUDS)  or  base  realignment  and  closure  (BRAC)  sites.  These  sites  must  be  cleaned  to  an 
appropriate  level  and  certified  as  suitable  for  their  intended  end  use. 

With  the  support  of  the  Environmental  Security  Technology  Certification  Program  (ESTCP) 
Project  UX-0031,  an  airborne  version  of  the  multisensor  towed  array  detection  system  (MTADS) 
vehicular  towed  array  has  been  developed  and  demonstrated.  The  objective  of  this  project  was  to 
produce  an  efficient  and  economical  UXO  survey  system  with  production  rates  and  costs 
appropriate  for  surveying  large  tracts  of  land.  While  the  system  we  developed  is  ideally  suited  to 
localizing  burial  caches  of  ordnance  and  establishing  which  areas  are  uncontaminated,  it  also 
retains  the  MTADS’  capability  of  detecting  and  locating  individual  ordnance  items  the  size  of 
2.75-in  rocket  warheads  (and  larger). 

The  system  deploys  a  linear  array  of  seven  Cs-vapor  magnetometers  spaced  at  1.5-m  intervals  in 
a  forward-mounted  boom.  The  system  is  certified  for  operation  on  all  models  of  the  Bell  Long 
Ranger  helicopter.  Two  global  positioning  system  (GPS)  units  mounted  on  the  forward  boom 
provide  positioning  and  helicopter  roll  and  yaw  measurements.  A  pilot  guidance  display 
provides  survey  progress  and  platfonn  infonnation  in  real  time.  The  data  acquisition  electronics 
rack  is  mounted  in  one  of  the  rear  seat  positions. 

1.2  DEMONSTRATION  RESULTS 
1.2.1  Badlands  Bombing  Range 

The  first  demonstration  was  at  the  Badlands  Bombing  Range  (BBR),  which  was  used  for  many 
years  for  ground  artillery  training  (105-mm,  155-mm,  and  8-in  projectiles).  The  airborne  system 
performance  was  evaluated  against  the  vehicular  MTADS  in  a  110-acre  survey  (which  included 
a  10-acre  area  where  inert  projectiles  were  blind  seeded).  All  targets  in  the  vehicular  and 
airborne  target  reports  were  dug.  The  Airborne  MTADS  then  surveyed  an  additional  1,600 
acres.  About  one  half  of  the  targets  in  this  target  report  were  also  dug.  In  the  commonly 
surveyed  areas,  the  vehicular  and  airborne  systems’  ordnance  detection  capabilities  were 
indistinguishable  from  one  another;  all  inert  and  live  UXO  detected  in  the  vehicular  survey  and 
analyses  were  also  detected  in  the  airborne  survey.  The  ability  to  distinguish  ordnance  from 
clutter  was  more  difficult  with  the  airborne  platfonn.  The  airborne  survey  analyses  contain  67% 
more  targets  than  the  vehicular  surveys.  The  high-density  data  in  the  vehicular  survey  enables 
many  non-UXO  targets  to  be  excluded  from  consideration  on  the  basis  of  shape  information  that 
is  not  available  in  the  much  sparser  airborne  data.  The  airborne  analyses  also  produce  priority 
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assignments  that  are  skewed  toward  the  priority  1,  2,  and  3  categories,  again  because  the  shape 
information  in  the  anomaly  signature  that  the  analyst  uses  is  not  present  to  the  same  degree  in  the 
airborne  data.  The  airborne  survey  production  rate  was  nearly  500  acres  per  survey  day. 

1.2.2  Aberdeen  Proving  Ground 

The  second  demonstration  was  at  the  Aberdeen  Proving  Ground  (APG)  on  five  sites  containing 
different  ordnance  types  and  densities.  Topographies  varied  from  flat,  level,  grass-covered 
meadows  to  trees  and  brush,  wetlands,  freshwater  ponds,  and  marine  offshore  areas.  Inert 
ordnance  was  seeded  into  three  of  the  sites,  including  one  area  that  had  not  previously  been  used 
as  a  range.  Detection  of  the  seed  targets  varied  from  very  good  on  the  airport  site  (85%  detection 
with  six  false  alarms  per  detection  to  94%  detection  with  11.5  false  alarms  per  target)  to  near 
zero  on  a  highly  cluttered  range.  Detection  of  ordnance  (81 -mm  and  105-inm)  was  difficult  in 
the  ponds  but  straightforward  in  the  offshore  areas  populated  by  larger  targets.  Extensive, 
preexisting  targets  were  dug  on  one  of  the  highly  cluttered  ranges;  more  than  30%  of  the 
recovered  targets  were  ordnance.  The  Airborne  MTADS’  performance  was  measured  against 
blind  seeded  targets  and  relative  to  another  airborne  survey  system  fielded  by  Oak  Ridge 
National  Laboratory  (ORNL).  The  Airborne  MTADS’  production  rate  on  these  small  sites  was 
about  35  acres/hour. 

1.2.3  Isleta  Pueblo 

The  third  demonstration  was  at  the  Isleta  Pueblo  in  New  Mexico  on  a  range  used  for  airborne 
training  during  the  1950s.  This  range  has  a  prominent  central  bull’s  eye,  which  was  populated 
by  a  high  density  of  buried  ordnance  and  ordnance-related  clutter.  Areas  north  and  south  of  the 
bull’s  eye  were  surveyed  by  the  vehicular  MTADS.  Small  (60-imn  and  81-mm)  and  medium 
(105-inm  and  155-inm)  seeded  targets  had  also  been  placed  in  these  100-acre  areas.  These  areas 
had  a  relatively  high  density  of  metallic  clutter  and  significant  geological  interferences.  The 
vehicular  survey  detection  capability  for  the  seeded  targets  (96%  detection  with  16  false  alarms 
per  hectare)  was  better  than  that  of  the  airborne  system  (64%  detection,  excluding  the  60-imn 
mortars  with  1 1  false  alarms  per  hectare).  The  Airborne  MTADS  and  the  airborne  ORNL  survey 
systems  each  surveyed  about  1,500  acres  centered  on  the  bull’s  eye.  Extensive  targets  were  dug 
from  these  target  reports.  The  Airborne  MTADS’  production  rate  on  this  desert  range 
approached  60  acres  per  hour. 

1.2.4  Cost  Assessment 

There  are  no  commercial  vendors  offering  airborne  UXO  geophysical  services.  We  estimate, 
based  on  our  production  rates  and  costs,  that  ultimately  the  production  costs  for  airborne  UXO 
search  services  will  likely  range  from  $100-$200/acre,  depending  on  the  site  size  and  conditions. 
The  Airborne  MTADS  is  appropriate  for  wide  area  searches  (at  least  500  acres,  i.e.,  at  least  1 
survey  day).  Many  sites  will  not  be  able  to  be  completely  characterized  using  the  airborne 
system,  however,  if  100%  coverage  is  required.  Most  sites  will  require  some  fill-in  work  by 
ground-based  systems. 
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1.3  REGULATORY  AND  STAKEHOLDER  ISSUES 


The  regulatory  issues  affecting  the  UXO  problem  are  most  frequently  associated  with  the  BRAC 
and  FUDS  processes  involving  the  transfer  of  DoD  property  to  other  government  agencies  or  to 
the  civilian  sector.  When  transfer  of  responsibility  to  other  government  agencies  or  to  the 
civilian  sector  takes  place,  the  DoD  lands  fall  under  the  compliance  requirements  of  the 
Superfund  statutes.  Section  2908  of  the  1993  Public  Law  103-160  then  requires  adherence  to 
Comprehensive  Environmental  Response,  Compensation,  and  Liability  Act  (CERCLA) 
provisions.  The  basic  issues  center  on  the  assumption  of  liability  for  ordnance  contamination  on 
previously  DoD-controlled  sites.  These  regulatory  considerations  do  not  apply  to  active  DoD 
facilities. 

The  Airborne  MTADS  is  an  appropriate  technology  for  addressing  the  UXO  problem  in  areas 
where  the  terrain  cannot  be  traversed  on  foot,  that  are  dangerous  for  ground  activities,  and  that 
are  too  large  to  survey  economically  with  vehicular  systems.  These  demonstrations  provide  data 
that  can  be  used  to  demonstrate  a  statistical  probability  of  success  for  the  detection  and 
characterization  of  isolated  bombing  targets  or  impact  areas,  ordnance  burial  caches,  or 
individual  ordnance,  including  a  range  of  medium-  and  large-sized  ordnance.  These 
considerations  are  important  in  establishing  the  value  of  this  approach  and  in  its  ultimate 
acceptance  by  regulators  and  the  stakeholder  community. 
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2.0  TECHNOLOGY  DESCRIPTION 

2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  Automated  Georeferenced  Surveys 

The  Strategic  Environmental  Research  and  Development  Program  (SERDP),  ESTCP,  and  the 
U.S.  Army  Environmental  Center  (AEC)  UXO  Advanced  Technology  Demonstration  Programs 
for  nearly  a  decade  have  been  addressing  the  need  for  more  modern,  automated  UXO  detection 
and  characterization  technologies.  These  investments  have  resulted  in  the  development, 
demonstration,  and  commercialization  of  automated  site  characterization  technologies  such  as 
MTADS.  The  original  MTADS  consists  of  a  tow  vehicle  and  two  low  self-signature  platfonns: 
one  for  an  eight-sensor  magnetometer  array,  the  other  for  a  three-sensor,  time-domain, 
electromagnetic  (EM)  pulsed-induction  array.1  MTADS  uses  GPS  for  recording  sensor  position 
locations  and  for  survey  guidance,  and  it  employs  a  sophisticated  data  analysis  system.  MTADS 
has  demonstrated  relatively  rapid  and  efficient  surveying  of  large  sites,  with  commensurate 
economic  benefits,  for  a  wide  range  of  buried  UXO  items  at  their  maximum  likely  penetration 
depths."'  On  ranges  with  relatively  uncomplex  use  histories  (i.e.,  ranges  involving  the  use  of 
similar  types  of  ordnance,  such  as  only  air-deployed  bombs  and  practice  bombs,  or  only  surface 
gun-fired  projectiles),  routine  UXO  detection  probabilities  of  greater  than  95%  are  often 
achieved  in  areas  without  severe  geological  interferences.  Use  of  fully  integrated  GPS  enables 
sensor  measurements  to  be  time-  and  location-stamped  so  that  the  survey  products  are 
georeferenced  digital  maps  of  the  survey  area  for  which  buried  target  signals  can  be  analyzed 
using  physics-based  fitting  algorithms.  The  survey  products  are  compatible  with  geographic 
information  system  (GIS)  mapping  technologies.  The  survey  results  can  thus  be  permanently 
archived,  used  for  quality  assurance  (QA)/quality  control  (QC)  evaluations,  organized  to  support 
subsequent  (including  delayed)  remediation  activities,  and  used  to  evaluate  or  defend  the 
performance  of  the  system  if  legally  challenged.  In  extended  surveys,  all  of  the  UXO  site 
characterization  activities,  including  the  surveying,  target  analysis,  and  preparation  of  reporting 
documents  to  support  remediation  activities,  can  be  delivered  for  $400-1,000  per  acre,  depending 
on  the  size  and  complexity  of  the  site.  The  MTADS  technology  (both  the  vehicular  and  man- 
portable  platforms)  was  transitioned  to  the  commercial  sector  (Blackhawk  Geometries,  Inc.)  by 
means  of  a  Cooperative  Research  and  Development  Agreement  (CRADA)9  and  is  currently 
being  used  to  provide  commercial  UXO  services  to  the  DoD. 

This  technology  has  provided  a  huge  step  forward  in  capability,  efficiency,  and  economy  for 
UXO  site  characterization.  The  DoD,  the  U.S.  Environmental  Protection  Agency  (EPA),10  and 
the  Army  Corps  of  Engineers  have  sanctioned  this  approach  as  the  preferred  technology  that 
should  be  used  by  default  unless  there  are  mitigating  circumstances.  While  this  has  been 
declared  the  technology  of  choice,  only  a  fraction  of  the  UXO  site  characterization  activities  are 
currently  being  carried  out  using  the  modem  technology.  There  are  purportedly  three  mitigating 
circumstances  justifying  the  continued  use  of  Mag  and  Flag  for  UXO  surveys.  These  include 
sites  that  are  too  small  to  justify  use  of  vehicular  systems,  sites  where  forest  canopies  or  limited 
sky  visibility  precludes  the  use  of  GPS,  and  sites  where  the  surface  geology  or  topology  is  not 
suitable  for  vehicular  surveys  and  that  are  too  small  for  cost-effective  airborne  surveys.  These 
three  limitations  have  been  addressed  by  the  man-portable  MTADS  adjuncts,  which  employ  both 
GPS  and  acoustic  navigation.  Under  ESTCP  Project  UX-9811  (Man-Portable  Adjuncts  for  the 
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MTADS),  NRL  developed  and  demonstrated  man-portable  adjuncts  to  the  vehicular  MTADS 
arrays.11'13  Both  the  magnetometer  and  EM  vehicular  systems  can  be  implemented  with  either 
GPS  or  acoustic  navigation  to  enable  surveying  in  areas  without  sky  view.  These  man-portable 
adjuncts  to  the  MTADS  have  also  been  transitioned  to  the  commercial  sector  through  the 
CRADA  with  Blackhawk  Geometries.9  Variants  of  the  NRL  man-portable  MTADS  hardware, 
as  demonstrated  for  ESTCP,  are  generally  available  from  several  commercial  UXO  service 
providers. 

One  significant  limitation  of  the  man-portable  systems  is  that  while  they  have  relatively  modest 
deployment  and  mobilization  costs,  they  invariably  are  more  expensive  to  operate  (on  a  per-acre 
basis)  than  the  vehicular  systems.  Man-portable  MTADS  survey  costs  are  typically  similar  to 
the  costs  of  Mag  and  Flag  UXO  survey  products.  Even  given  this  limitation,  use  of  the  man- 
portable  MTADS  is  preferable  because  it  provides  digitally  referenced  survey  products. 

For  very  large  sites  where  the  costs  associated  with  UXO  surveys  formerly  precluded  any 
comprehensive  action  from  being  undertaken,  the  Airborne  MTADS,  described  below,  has 
become  a  low-cost,  high  production  rate  option. 

2.1.2  The  Airborne  System 

The  Naval  Research  Laboratory  (NRL),  with  the  support  of  ESTCP  Project  UX-0031,  has 
adapted  the  vehicular  MTADS  magnetometry  technology  for  deployment  on  an  airborne 
platform.14  The  primary  objective  of  this  development  was  to  provide  a  UXO  site 
characterization  capability  for  extended  areas  that  are  inappropriate  for  vehicular  or  man- 
portable  surveys.  Because  the  sensors  on  an  airborne  platfonn  must  be  deployed  farther  from  the 
ground  surface  than  those  on  vehicular  or  man-portable  systems,  it  is  understood  that  detection 
sensitivity  for  single,  smaller  UXO  items  is  compromised.  It  has  been  a  goal  of  the 
development,  however,  to  retain  as  much  detection  sensitivity  as  possible  for  individual  UXO 
targets. 

Sites  appropriate  for  airborne  surveys  include  those  with  terrain  that  would  be  difficult  to  survey 
efficiently  with  a  vehicular  system  and  those  that  are  too  extensive  to  evaluate  economically  with 
vehicular  or  other  approaches.  Many  formerly  used  ranges  dating  from  World  War  II  (and 
earlier)  are  located  in  areas  involving  tens  or  hundreds  of  thousands  of  acres  with  isolated 
bombing  targets  or  impact  ranges.  Locations  of  many  of  these  impact  areas  (or  ordnance  burial 
caches)  are  either  not  known  or  imprecisely  known.  Some  of  these  areas  are  located  on  Native 
American  reservations,  while  others  involve  Closed,  Transferred  or  Transferring  (CTT)  ranges. 
Therefore,  an  additional  objective  of  the  development  was  that  the  final  airborne  system  have 
survey  production  rates  and  costs  appropriate  for  exploring  very  large  sites  that  would  be 
prohibitively  expensive  to  survey  by  other  techniques. 

2.1.3  System  Specifications  and  Requirements 

It  was  realized  during  our  initial  modeling  studies  that  by  using  magnetometer  arrays  mounted  on 
helicopter  platforms,  the  smallest  military  ordnance  would  not  be  detectable  as  individual  targets. 
Modeling  calculations  were  carried  out  to  evaluate  target  signatures  as  a  function  of  altitude  (i.e., 
the  standoff  distance  between  the  target  and  sensor).  Helicopter  pilots  were  interviewed  to 
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determine  the  practical  flying  limitations  for  altitude,  payload,  platform  design,  and  mission 
endurance  that  could  be  expected.  We  developed  and  refined  the  specifications  and  requirements 
that  became  part  of  our  original  proposal  and  the  development  plan.  Table  1  shows  a  summary 
of  the  design  specifications  from  the  requirements  document  in  the  Airborne  MTADS 
development  plan. 

Table  1.  System  Specifications  and  Requirements  for  the  Airborne  MTADS. 


Activity 

Requirement 

Survey  flight  duration 

2  hours  (including  ferry  and  calibration  time) 

Survey  speed 

10-20  m/sec 

Lane  spacing 

7.5  m  (nominal)  * 

Survey  area  (single  setup) 

250  acres 

Flights  per  day 

3  (single  pilot) 

Detection  sensitivity 

Isolated  BDU-33  or  2.75-in  warheads 

Sensor  sensitivity 

0.01  nT 

Sensor  data  rate 

100  Hz 

GPS  navigation  data  rate 

20  Hz 

GPS  sensor  position  accuracy 

5  cm 

Data  acquisition  system  (DAQ) 

Compatible  with  vehicular  MTADS  DAQ 

Data  analysis  system  (DAS) 

Seamless  integration  with  vehicular  data 

*Depending  on  winds  and  pilot  experience 


2.1.4  Field  Hardware 

The  Airborne  MTADS  hardware  incorporates 
an  array  of  seven  magnetometers  on  a 
platform  designed  for  a  Model  206L  Bell 
Ranger  helicopter.  The  sensors  are  Cs-vapor, 
full-field  magnetometers,  Geometries  Model 
822A.  The  specially  selected  magnetometers 
are  airborne  quality.  The  helicopter  with  the 
mounted  magnetometer  array  is  shown  in 
Figure  1 .  All  sensors  are  interfaced  to  the  data 
acquisition  system  (DAQ)  computer.  The 
DAQ  electronics  are  contained  in  a  rack 
mounted  in  the  rear  starboard  seat  position  in 
the  helicopter.  The  power  distribution 
interface  is  also  in  the  rack,  as  are  readouts  for 
all  the  sensor  inputs.  On  the  9-m  boom,  the 
seven  sensors  are  mounted  with  a  1.5-m 
horizontal  spacing.  The  sensor  positions  over 
the  surface  of  the  Earth  (latitude,  longitude,  and  height  above  ellipsoid  [HAE])  are  determined 
using  satellite-based  GPS  navigation,  employing  the  latest  real-time  kinematic  (RTK) 
technology,  which  provides  a  real-time  position  update  (at  20  Hz)  with  an  accuracy  in  the 
horizontal  plane  of  about  5  cm.  GPS  satellite  clock  time  is  used  to  time-stamp  both  position  and 
sensor  data  information  for  later  correlation. 


Figure  1.  Airborne  MTADS  Survey  on  the 
Active  Recovery  Field.  (Note  the  2-m-high 
vegetation  that  stretches  from  this  point  to  the 
shoreline.) 
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Dual  GPS  antennas  (Trimble  Zephyrs),  deployed  on  the  forward  horizontal  boom,  in  addition  to 
providing  the  position  over  ground  and  the  height  above  ellipsoid  positions  for  sensor  mapping, 
provide  boom  roll-and-yaw  attitude  information  for  sensor  location  corrections.  An  inclinometer 
provides  the  pitch  attitude  correction,  and  a  fluxgate  gradiometer  provides  three-axis  information 
that  can  be  used  to  derive  aeromagnetic 
compensation  corrections  for  the  magnetometer 
sensor  data.  Laser  (Optech  Sentinel,  Model 
3100DV)  and  radar  (Terra,  Model 
TRA350/TRI40)  altimeters  mounted  on  fixtures 
attached  to  the  rear  hardpoint  of  the  helicopter 
provide  two  independent  altitude  measurements 
to  the  DAQ  computer.  The  dual  altimeters  were 
deployed  to  provide  complementary  information 
when  operating  over  water  or  vegetation. 

The  helicopter  pilot  flies  the  survey  using  an 
onboard  navigation  guidance  display  developed 
specifically  for  this  application.  The  sunlight- 
readable  screen  is  mounted  to  the  right  of  the 
instrument  panel,  as  shown  in  Figure  2,  so  it  is 
in  the  field  of  view  of  the  pilot  without  reducing  his  ability  to  visualize  the  whole  forward  boom 
and  the  field  immediately  ahead  of  the  helicopter.  The  survey  parameters  are  set  up  in  the  pilot 
display  computer.  This  computer  shares  the  navigation  and  altimeter  data  with  the  DAQ 
computer. 

The  navigation  guidance  display  provides  a  left-right  indicator,  an  altitude  display,  an  automatic 
line  number  increment,  an  adjustment  for  lateral  offset,  and  a  color-coded  flight  swath  overlay, 
with  the  ability  to  zoom  the  presentation  scale  in  or  out  on  the  display.  The  survey  course  over 
ground  (COG)  is  plotted  for  the  pilot  in  real  time  on  the  display,  as  are  presentations  showing  the 
laser  altimeter  data  and  the  GPS  navigation  fix  quality. 

2.2  PROCESS  DESCRIPTION 

2.2.1  Personnel  Training 

Shakedown  exercises  were  an  integral  part  of  preparing  for  demonstrations.  Once  the  airborne 
demonstrations  began  at  the  BBR,  they  took  place  almost  flawlessly.  This  is  in  contrast  to  our 
experience  during  the  shakedown  surveys.  There  were  three  shakedown  exercises  at  the  Airfield 
separated  by  1 -month  periods,  each  dominated  by  equipment  breakdowns,  malfunctions,  and 
misadventures.  We  recovered  and  fixed  most  of  the  mistakes  resulting  from  each  exercise  before 
the  next  shakedown.  These  shakedown  exercises  were  critical  to  the  success  of  the  final 
demonstrations.  It  was  important  that  they  be  separated  by  at  least  a  month  to  enable  us  to 
evaluate  problems,  order  parts,  implement  fixes,  and  plan  for  the  following  test. 

For  the  BBR,  APG,  and  Isleta  demonstrations,  pilot  orientation  flights  were  made  prior  to  each 
demonstration.  The  pilot  was  requested  to  fly  at  the  lowest  altitude  consistent  with  flight  safety. 


Figure  2.  The  Navigation  Guidance  Display  Is 
Mounted  on  the  Starboard  Side  of  the  Cockpit 
for  the  Pilot's  Use  During  Surveys. 
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2.2.2  Data  Processing 


Survey  and  navigation  data  recorded  in  the  DAQ  computer  are  transferred  (using  a  Zip  disk,  a 
“memory  stick,”  or  a  notebook  computer)  to  the  DAS  computer.  The  DAS  software  was 
developed  specifically  for  the  MTADS  (vehicular,  man-portable,  and  Airborne)  as  a  standalone 
suite  of  programs,  written  using  Interactive  Data  Language  (IDL)  development  tools,  and 
graphical  user  interfaces  (GUI)  working  in  a  UNIX-based  workstation  environment.  Over  a 
period  of  about  2  years,  the  MTADS  DAS  was  adapted  to  operate  in  a  Windows™  environment 
on  a  PC. 


The  first  task  of  the  analyst  is  inspection  and 
processing  of  the  data  in  preparation  for  target 
analysis.  Initially,  files  are  reviewed  to  determine 
sensor  data  quality.  Necessary  edits  are  carried 
out  to  remove  spurious  sensor  readings  to  clean 
up  the  navigation  files.  The  background  readings 
for  all  the  sensors  in  the  array  are  leveled  to  null 
sensor  offsets.  Glitches  in  the  GPS  navigation  are 
corrected  using  the  COG  displays.  Small  offsets 
often  occur  when  the  mix  of  satellites  used  in  the 
solution  changes.  Typically,  a  1,000-point,  down- 
the-track  demedian  filter  is  applied  to  correct  for 
directional  and  platform-induced  errors  and  for 
large-scale  geological  interferences.  The 
navigation  and  sensor  files  are  then  processed 
together  to  establish  a  three-dimensional  (3-D) 
coordinate  location  for  each  magnetometer  sensor 
reading.  Finally,  the  individual  survey  files  are  assembled  into  site  survey  maps  (mapped  data 
files).  At  this  point,  target  analysis  can  begin.  Historically,  these  operations  have  been  carried 


Figure  3.  Working  Screen  of  the  MTADS  DAS 
Showing  the  Survey  Project  View  on  the  Left  and 
an  Expanded  Analysis  Window  on 
the  Right. 


out  using  utilities  associated  with  the  MTADS  DAS. 
Figure  3. 


A  working  screen  of  the  DAS  is  shown  in 


The  DAS  employs  resident  physics-based  models  to  determine  target  size,  position,  and  depth. 
Extensive  data  sets  have  been  acquired  and  processed  to  calibrate  the  models.  Using  these 
models,  we  have  demonstrated  target  location  accuracies  of  ~  15  cm  with  the  magnetometer 
system. 


Although  we  have  achieved  impressive  results  using  the  DAS,  it  has  proven  difficult  to  transition 
the  analysis  utility  to  the  general  UXO  user  community.  After  the  BBR  demonstration,  we 
began  performing  the  data  processing  functions  by  generating  mapped  data  files  using  a 
commercial  software  utility,  Geosoft’s  Oasis  montaj™. 
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2.2.3  Data  Analysis 


Currently,  we  create  mapped  data  fdes  using 
either  Oasis  montaj™  or  the  MTADS  DAS. 

For  target  selection  and  analysis,  we 
currently  use  the  MTADS  DAS.  Under  a 
separate  program,  we  are  in  the  process  of 
converting  the  analysis  routines  developed 
under  ESTCP  and  SERDP  sponsorship  to 
Geosoft  Executables  (GX),  executable  fdes 
that  can  be  called  from  the  Oasis 
environment.  Ultimately,  this  will  enable 
the  analyst  to  perform  all  of  the  data  analysis 
entirely  within  the  Oasis  environment.  All 
target  analyses  reported  in  this  document 
were  accomplished  using  the  MTADS  DAS. 

The  MTADS  target  analysis  GUI  is  written 
at  multiple  levels  to  accommodate  both 
sophisticated  and  novice  users.  A  novice 
user  can  perform  data  analysis  using  menu- 
driven  tools  and  the  background  default  analysis  settings  (see  Figure  4).  When  a  magnetic 
anomaly,  such  as  the  one  shown  in  Figure  4,  is  boxed  for  analysis  using  the  computer  mouse,  the 
DAS  selects  the  sensor  data  within  the  boxed  area  for  consideration.  Each  sensor  reading,  with 
its  HAE,  is  an  input  datum  used  in  the  seven-parameter  iterative  calculation  to  produce  the  best 
fit  to  a  dipole  model  of  the  anomaly  signature.  Extensive  training  data  sets  (using  inert 
ordnance)  have  been  used  to  refine  the  algorithms  to  improve  target  analysis.  In  addition  to 
position,  depth,  and  size  solutions,  magnetic  analyses  provide  dipole  orientation  and  effective 
target-caliber  information  and,  using  a  “goodness  of  fit”  analysis,  provide  guidance  in  the  target- 
fitting  process. 

2.3  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

The  Airborne  MTADS  system  was  extensively  tested  and  improved  as  the  result  of  the  three 

shakedown  tests  conducted  at  the  Aberdeen  Proving  Ground  Airfield.  For  further  information, 

see  “Airborne  MTADS  Demonstration  on  the  Impact  Area  of  the  Badlands  Bombing  Range: 
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Technology  Demonstration  Plan”. 

2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

Unlike  the  vehicular  magnetometer  system,  the  airborne  system  is  not  capable  of  detecting  the 
smallest  classes  of  buried  UXO  at  depth.  While  the  magnetic  anomaly  signals  are  spatially 
spread  and  diminished  in  intensity  with  the  sensors  farther  above  the  ground,  our  modeling 
results  indicated  that,  at  an  altitude  of  2  m  above  the  ground,  the  system  should  be  capable  of 
detecting  bomb  unit  dispensed  (BDU)-33s  or  Mk  82s  in  all  geologies  and  ordnance  targets 
equivalent  to  or  larger  than  2.75-in  warheads  in  geologically  quiet  areas.  This  has  generally  been 


Figure  4.  Site  View  and  Data  Analysis  Screens  from 
the  MTADS  Data  Analysis  Program.  (A  part  of  the 
Mine,  Grenade,  and  Direct-Fire  Weapons  Range  survey  is 
shown  on  the  left.  An  individual  target  is  boxed  for 
analysis  on  the  right.) 


10 


borne  out  by  the  demonstrations  described  in  this  report.  At  the  geologically  quiet  and 
topologically  flat  prove-out  site  at  the  Airfield,  we  efficiently  detected  both  60-mm  and  81 -mm 
mortars.  ’  At  the  much  more  highly  cluttered  and  geologically  active  Isleta  range,  in  areas 
with  rough  ground  surface  or  significant  vegetation,  we  failed  to  detect  several  105-mm 
projectiles.17'23 

The  extent  to  which  spreading  target  signatures  interfere  with  each  other  and  are  obscured  by 
geological  features  was  carefully  evaluated  in  the  first  airborne  demonstration  at  the  BBR.15  In 
that  study,  with  fairly  large  UXO  targets  (105-mm  to  8-in  projectiles)  relatively  sparsely 
distributed  on  the  site,  detection  efficiency  for  individual  UXO  was  equivalent  for  the  airborne 
and  vehicular  towed  arrays.  Because  of  the  lower  data  density  and  the  more  widely  spread 
anomaly  signatures,  it  proved  more  difficult  to  discriminate  between  UXO  and  clutter  signatures 
from  the  airborne  data  than  from  the  vehicular  data.  At  some  APG  sites,16  significantly  more 
targets  would  have  to  be  dug  behind  an  airborne  survey  than  behind  a  corresponding  vehicular 
survey.  This  is  necessitated  by  the  much  higher  target  densities  and  the  more  complex  mix  of 
UXO  threats  on  some  of  these  ranges  that  result  in  merging  and  overlapping  of  adjacent  target 
signatures.  The  cost  tradeoffs  between  digging  more  targets  and  reduced  survey  production  costs 
are  (and  will  always  be)  site  specific,  depending  on  the  types  of  UXO  challenges,  the  relative 
density  of  targets,  geological  and  topological  conditions,  and  the  size  of  the  survey  site. 
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3.0  BADLANDS  BOMBING  RANGE  DEMONSTRATION  DESIGN 

3.1  PERFORMANCE  OBJECTIVES 

The  objectives  of  this  demonstration  are  enumerated  below: 

•  Prepare  a  10-acre  area  seeded  with  25  ordnance  items  whose  locations  are  unknown  to 
the  survey  team.  Survey  the  seeded  area  and  an  additional  100  acres  with  the  vehicular 
MTADS  magnetometer  array. 

•  Complete  an  extended  site  survey  that  includes  the  seeded  area,  the  additional  100-acre 
prove-out  area,  and  other  accessible  parts  of  the  impact  area  (IA)  using  the  Airborne 
MTADS.  (The  airborne  survey  was  planned  to  cover  about  1,700  of  the  2,400-acre  IA.) 

•  Based  on  on-site  target  analysis  of  the  data,  use  a  UXO-certified  recovery  team  to  dig  all 
targets  on  the  survey  dig  list  from  the  seeded  area  and  the  100-acre  survey  area. 

•  Based  on  the  dig  list  prepared  from  the  airborne  survey  of  the  remaining  area,  dig  targets 
from  the  area  surveyed  only  by  the  airborne  system,  beginning  with  the  highest  priority 
and  continuing  until  funds  are  exhausted. 

•  Provide  the  survey  products  and  reports  to  the  BBR  Project  Office  of  the  Oglala  Sioux 
Tribe  (OST). 

•  Prepare  a  report15  of  our  activities,  which  includes  a  description  of  all  dug  targets,  a 
listing  of  the  positions  and  descriptions  of  all  targets  observed  in  the  survey  that  were  not 
dug,  and  an  evaluation  of  the  airborne  system’s  perfonnance. 

3.2  PERFORMANCE  METRICS 

3.2.1  Target  Location  Accuracy 

A  10-acre  area  was  seeded  with  25  ordnance  items,  and  the  ground  truth  for  the  site  was  held  by 
the  U.S.  Anny  Engineer  Research  and  Development  Center  (ERDC)  until  after  the 
magnetometry  analyses  from  both  the  vehicular  and  airborne  surveys  had  been  submitted  to 
ESTCP  and  ERDC.  Comparisons  between  the  systems  were  made,  as  well  as  evaluations  of  the 
absolute  target  location  accuracies. 

3.2.2  System  Operational  Performance 

Field  logs,  kept  for  all  survey  activities,  provided  infonnation  about  setup  times,  survey  times, 
production  rates,  and  equipment  performance.  In  preparing  for  each  day’s  operations,  specific 
attention  was  given  to  the  GPS  satellite  availability  schedule.  Field  notes  were  made  about  the 
operational  perfonnance  of  the  Airborne  MTADS,  specifically  documenting  special  weather 
conditions  or  areas  of  difficult  topography  or  vegetation  that  either  influenced  system 
performance  or  forced  missed  areas  that  could  not  be  recovered  by  reflying. 

The  electronic  data  files  provided  additional  information  about  field  survey  performance  by 
documenting  survey  data  collection  times,  the  course  used  in  data  collection,  stoppages  during 
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data  collection,  how  turnarounds  were  accomplished,  and  how  well  pilots  lined  up  to  begin  their 
survey  paths. 

3.2.3  Detection  Capability 

This  demonstration  was  the  first  use  of  the  Airborne  MTADS  to  conduct  an  extensive  survey. 
We  compared  the  relative  abilities  of  the  vehicular  and  the  airborne  systems  to  detect  the  seeded 
ordnance  and  to  differentiate  between  clutter  and  ordnance.  On  the  IA,  the  Airborne  MTADS 
hardware  performed  flawlessly  in  the  field,  and  the  data  processing,  analysis,  and  target  picking 
performance  was  routine,  exceeding  our  expected  production  rates. 

3.3  SITE  SELECTION 

MTADS  demonstration  projects  during  the  period  1996-2001  were  sponsored  primarily  by 
ESTCP  and  the  U.S.  Army  Corps  of  Engineers,  Huntsville  Engineering  and  Support  Center3'4 
(CEHNC).  With  the  exception  of  a  study  of  UXO  contamination  on  the  beach  at  the  former  Fort 
Pierce  Naval  Amphibious  Training  Base,  the  MTADS  demonstrations  have  focused  on  ranges 
impacted  by  bombing  and  aerial  and  ground  gunnery  training.  In  1999,  we  conducted  a 
vehicular  MTADS  survey  of  a  small  portion  of  the  IA  at  the  BBR.17  In  preparation  for  that 
project,  NRL  conducted  site  visits  and  archival  records  searches,  coordinated  OST  activities, 
acquired  aerial  photography,  and  established  first-order  control  points  to  support  the  survey.  The 
1999  survey  also  supported  this  and  subsequent  demonstrations. 

In  September  2001,  we  returned  to  the  same  range  to  complete  the  Advanced  UXO 
Classification  Demonstration  with  the  vehicular  MTADS  (a  different  ESTCP  project)  and  to 
conduct  the  first  demonstration  of  the  Airborne  MTADS  adjunct  platfonn.  In  support  of  both  of 
these  ESTCP  project  demonstrations,  a  10-acre  site  was  seeded  with  25  degaussed  targets — five 
8-in,  ten  155-inm,  and  ten  105-mm  projectiles.  See  Figure  5.  This  area  was  surveyed  with  the 
vehicular  MTADS  EM  array.  Subsequently,  this  site  and  an  additional  100  acres  were  surveyed 
with  the  vehicular  MTADS  magnetometer  array  prior  to  beginning  the  Airborne  MTADS  survey. 

3.4  TEST  SITE  HISTORY  AND  CHARACTERISTICS 

In  1942,  the  Department  of  War  annexed  341,725  acres  of  the  Pine  Ridge  Reservation  for  use  as 
an  aerial  gunnery  and  bombing  range.  This  site  is  located  in  southwestern  South  Dakota,  with 
the  largest  part  of  the  Badlands  Bombing  Range  located  in  Shannon  County.  From  1942  until 
1948,  various  sections  of  this  range  were  used  for  bombing  exercises  and  air-to-ground 
operations.  Since  1960,  portions  of  the  land  have  been  returned  to  the  OST  in  a  stepwise 
fashion.  In  1968,  Congress  enacted  Public  Law  90-468,  returning  202,357  acres  to  the  OST  and 
setting  aside  136,882  acres  of  formerly  held  OST  lands  to  form  the  Badlands  National 
Monument,  to  be  managed  by  the  National  Park  Service.  In  1978,  all  remaining  BBR  lands  were 
declared  excess  federal  property  with  the  exception  of  2,486  acres  (subsequently  referred  to  as 
the  Air  Force  Retained  Area  or  the  Impact  Area).  Circa  1965,  the  South  Dakota  National  Guard 
placed  as  many  as  100  car  bodies  on  the  2,486-acre  area  and  began  using  them  as  ground-to- 
ground  artillery  targets  during  training  exercises.  The  National  Guard  training  exercises  took 
place  on  the  I A  between  1966  and  1973. 
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Figure  5.  The  BBR  Impact  Area  (within  the  red  boundary),  the  1999  Survey 
Area  (shown  in  green),  and  the  2001  10-Acre  Seed  Target  Area  (bounded  by  a 
thinner  red  border)  Within  This  Area. 

There  have  been  six  UXO  clearance  operations  carried  out  on  the  BBR  between  1948  and  1997. 
These  are  discussed  in  more  detail  in  Reference  28.  Only  two,  in  1975  and  1996,  have 
significant  relevance  to  the  present  demonstration  on  the  IA.  No  record  of  air-to-ground 
exercises  exists  that  specifies  the  IA  as  a  target.  During  the  summer  and  fall  of  1975,  10 
Explosive  Ordnance  Detection  (EOD)  personnel  participated  in  a  walking  search  line  clearance 
of  22,403  acres  and  a  vehicular  search  of  19,222  acres.  This  included  a  walking  search  line 
survey  of  the  entire  IA  and  the  buffer  zone.  With  the  exception  of  the  IA,  all  lands  were  declared 
as  cleared  and  certified  for  return  to  the  OST.  The  IA  reportedly  contained  too  much  ordnance 
and  explosive  (OE)  material  to  declare  the  area  cleared. 

During  the  4-month  summer  period  in  1996,  a  walking  and  driving  search  line  ordnance 
clearance  was  conducted  by  20  EOD  personnel  from  Ellsworth  Air  Force  Base  (AFB).  With  the 
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exception  of  56  acres  of  rugged  terrain  along  the  White  River  escarpment,  the  entire  I A  was 
covered.  EOD  teams  used  metal  detectors  to  clear  the  area  to  a  depth  of  1.5  ft.  The  OE  scrap 
recovered  included  4,000  lbs  of  shrapnel  (pieces  larger  than  3  in)  and  an  additional  8,000  lbs  of 
non-ordnance-related  metal  scrap. 

3.5  PHYSICAL  SETUP  AND  OPERATIONS 

Support  for  the  MTADS  demonstration  on  the  IA  was  provided  by  the  ESTCP.  Oversight  of  the 
NRL  activities  on  the  IA  was  provided  by  the  Environmental  Office  (Civ  28  CES/CEVR)  of 
Ellsworth  AFB.  All  operations  were  coordinated  with  the  ESTCP  Program  Office,  Corps  of 
Engineers  Northwestern  Division  Omaha  District  (CENWO),  Ellsworth  AFB,  EPA  (Region  8), 
the  South  Dakota  Department  of  Environment  and  Natural  Resources,  and  the  Badlands 

Bombing  Range  Project  Office  of  the  OST.  The  specific  operations  are  described  in  the 
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demonstration  test  plan. 

NRL,  Code  6110,  was  the  manager  for  all  activities  associated  with  the  Airborne  MTADS 
demonstration  on  the  IA.  The  NRL  on-site  project  manager,  Dr.  J.R.  McDonald,  was  responsible 
for  coordinating  operations  at  the  IA  and  approving  alterations  or  changes  to  the  demonstration 
plan  or  schedule.  All  persons  working  on  site  were  NRL  employees,  contractors  working  for 
NRL,  or  employees  or  subcontractors  of  the  prime  contractors. 

3.5.1  The  Seed  Target  Area 

APG  degaussed  inert  ordnance  to  prepare  the  seed  target  area.  The  ordnance  was  shipped  to 
ERDC  in  Vicksburg,  Mississippi,  and  was  transported  from  there  to  the  IA  in  South  Dakota 
where  the  10-acre  seeded  site  was  prepared  during  August  2001.  NRL  defined  the  comers  of  the 
test  site  and  provided  the  coordinates  to  ERDC.  The  boundary  of  the  10-acre  (200  m  x  200  m) 
area  is  indicated  by  the  thinner  red  outline  in  Figure  5.  The  10  acres  fall  primarily  in  an  area  that 
was  surveyed  in  1999  using  the  vehicular  magnetometer  array  and  was  subsequently  remediated 
to  remove  targets  that  were  potentially  105-mm,  155-mm,  or  8-in  projectiles.  Because  the  area  is 
only  100-300  m  away  from  the  bull’s  eye,  there  is  a  relatively  high  density  of  shrapnel  and 
clutter  present  on  the  site.  The  ground  truth  for  the  site  (Table  2)  was  held  by  ERDC  until  after 
the  magnetometry  analyses  from  both  the  vehicular  and  airborne  surveys  had  been  submitted  to 
ESTCP  and  ERDC. 

3.5.2  Logistics 

Because  of  the  complexity  caused  by  the  simultaneous  parallel  airborne  demonstration  and  the 
Advanced  UXO  Classification  Demonstration,  it  was  important  that  the  logistics  support  be 
carefully  planned  and  coordinated.  The  logistics  facilities  served  as  a  focal  point  for  all  field 
activities. 

Figure  6  shows  some  of  the  logistics  support  equipment  that  was  set  up  for  the  demonstration. 
The  leftmost  trailer  served  as  the  command  center.  All  data  analysis  computers  were  housed 
there.  The  next  trailer  provided  storage  for  the  hardware  and  housed  all  the  battery-charging 
stations.  The  third  trailer  was  the  site  office  for  the  OST  workers  and,  during  the  excavation 
operations,  also  for  the  UXO  teams.  The  fourth  trailer,  which  opened  at  both  ends,  served  as  a 
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drive-through  garage  for  the  vehicular  systems.  Between  the  fourth  trailer  and  the  tractor-trailer, 
a  tent  cover  was  set  up  to  provide  protection  for  working  on  the  vehicles  or  other  equipment. 
The  tractor-trailer  was  used  to  transport  the  vehicular  equipment  and  the  airborne  sensor 
platform  to  the  site.  The  truck  at  the  south  end  was  the  Jet  A  tanker  for  the  helicopter.  To  the 
east  of  this  equipment  were  located  portable  toilets,  a  65  kW  generator,  and  a  diesel  storage  tank. 
Not  shown  in  the  image  are  the  two  four-wheel-drive  backhoes  that  supported  the  UXO 
excavation  work. 

Table  2.  Ground  Truth  Table  for  the  Inert  Seed  Ordnance  Emplaced  at  the  Impact  Area. 


|  Item  # 

Northing  (m) 

Easting  (m) 

Depth  (m) 

Azi.  (deg) 

Incl.  (deg) 

Nose  U/D 

Serial  No.  | 

|  8-inch 

1-2 

4838171.34 

722824.74 

0.75 

350 

75 

D 

4 

1-4 

4838142.67 

722957.56 

0.50 

270 

45 

D 

5 

1-6 

4838117.82 

722874.46 

0.75 

40 

80 

D 

3 

1-8 

4838082.55 

722834.30 

0.30 

10 

0 

H 

6 

1-10 

4838019.39 

722889.76 

0.50 

340 

40 

D 

2 

|  155-mm 

1-12 

4838120.88 

722786.50 

0.85 

0 

45 

D 

10 

1-14 

4838086.48 

722802.76 

0.25 

250 

65 

D 

8  i 

1-16 

4838176.31 

722813.27 

0.60 

15 

80 

D 

12 

1-18 

4838143.69 

722819.03 

0.85 

115 

45 

D 

11 

1-20 

4838066.32 

722848.56 

0.25 

165 

70 

D 

13 

1-22 

4838142.69 

722860.13 

0.25 

110 

0 

H 

15 

1-24 

4838168.67 

722886.90 

0.30 

360 

35 

D 

9 

1-26 

4838106.46 

722901.24 

0.55 

75 

45 

U 

14 

1-28 

4838202.03 

722921.32 

0.60 

30 

40 

D 

6 

1-30 

4838137.07 

722919.42 

0.40 

310 

55 

D 

7 

|  105-mm  | 

1-32 

4838196.19 

722853.42 

0.25 

110 

35 

D 

16 

1-34 

4838176.23 

722831.42 

0.92 

05 

75 

D 

9 

1-36 

4838174.21 

722879.23 

0.40 

115 

45 

D 

10 

1-38 

4838164.65 

722931.82 

0.25 

30 

0 

H 

7 

1-40 

4838141.72 

722893.58 

0.50 

50 

55 

D 

13 

1-42 

4838118.78 

722830.47 

0.60 

245 

75 

U 

15 

1-44 

4838070.04 

722926.09 

0.50 

65 

60 

D 

12 

1-46 

4838064.41 

722957.64 

0.25 

315 

80 

D 

11 

1-48 

4838050.93 

722914.61 

0.30 

25 

35 

D 

8 

1-50 

4838032.77 

722808.48 

0.30 

360 

45 

D 

14 

U  -  up 

D  -  down 

H  -  horizontal 


3.5.3  On-Site  Support 

Two  NRL  employees  were  on  site  at  all  times  during  operations.  Dr.  J.R.  McDonald  was  the 
principal  investigator  (PI)  and  on-site  manager  for  the  Airborne  Demonstration  Project.  Dr.  H.H. 
Nelson  was  the  PI  and  on-site  manager  for  the  Advanced  UXO  Classification  Demonstration. 
Nova  Research,  Inc.,  coordinated  all  rentals  and  leases  for  on-site  equipment.  The  site  safety 
officer  was  an  EOD-certified  Nova  employee  who  also  had  responsibility  for  site  hardware 
maintenance  and  vehicle  operation.  AETC  Incorporated  supported  the  demonstrations  with  five 
on-site  employees.  They  supported  the  data  collection  and  processing  for  both  projects. 
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Additionally,  they  supported  the  Advanced  UXO  Classification  Demonstration’s  field  activities, 
managed  flight  operations  for  the  airborne  survey,  perfonned  data  analysis,  and  created  survey 
products.  Helicopter  Transport  Services,  Inc.,  provided  the  helicopter  and  pilot  for  the  airborne 
survey. 

Vehicular  survey  operations  were  supported  by 
three  to  five  OST  members  from  the  BBR  Project 
Office.  Additionally,  two  EOD  technicians  from 
the  BBR  Project  Office  supported  the  dig  teams. 

All  target  way  pointing  and  recovery  operations 
were  the  responsibility  of  Explosive  Ordnance 
Technology,  Inc.  (EOTI).  The  four-person  EOTI 
staff  and  the  certified  OST  technicians  formed 
two  dig  teams.  These  teams  conducted  all  target 
recovery  operations,  recorded  the  results  of  each 
dig  on  the  dig  sheets,  photographed  the  recovered 
objects  from  each  hole,  and  refilled  and  tamped 
each  hole,  returning  it  to  grade.  EOTI  was 
responsible  for  providing  explosives  and  blowing 
in  place  all  recovered  ordnance.  All  recovered 
OE  scrap  (and  other  metal  scrap)  was  certified  as 
explosives-free  and  stockpiled  for  disposal  by  Ellsworth  AFB. 

3.5.4  Demonstration  Activity 

The  spare  assemblies  for  the  airborne  platform  were  shipped  by  motor  freight  for  storage  at  the 
Rapid  City  Regional  Airport.  Since  they  were  not  needed,  they  remained  unopened  and  were 
returned  to  NRL.  All  the  other  equipment  was  shipped  in  a  5 3 -ft  tractor  trailer  that  left  Blossom 
Point  on  August  3 1  for  the  IA.  A  Nova  employee  arrived  at  the  IA  the  week  of  September  3  to 
oversee  the  placement  of  the  logistics  support  rental  equipment.  The  activity  log  in  Table  3 
describes  field  activities  of  each  of  the  project  demonstrations.  The  equipment  difficulties  with 
the  EM  sensors  required  adjusting  the  schedules  for  the  vehicular  survey  operations  to  allow  for 
repairs  and  recalibration  of  the  EM  array. 

The  September  11  terrorist  attacks  in  New  York  and  Washington  delayed  for  1  week  the 
departure  of  the  helicopter  from  Baltimore  to  support  our  operations.  It  was  uncertain  until 
September  20  whether  or  not  we  would  be  able  to  conduct  any  airborne  survey  activities. 


Figure  6.  Logistics  Setup  Supporting  the 
Demonstrations  at  the  Impact  Area. 
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Table  3.  Activity  Log  for  the  Demonstration  Projects  on  the  IA. 


Date 

Activity 

Result 

Comment 

5  Sep 

Logistics  support 

All  components  in  place 

Electrical  wiring  complete 

6  Sep 

Trailer  truck  arrives  on  site 

Backhoe  used  to  repair  road 

7  Sep 

MTADS  components  unpacked 
and  assembled 

9  Sep 

NRL  and  support  contractors 
arrive 

10  Sep 

Coordinate  OST  and  contractor 
activities 

Set  up  data  analysis  trailer 
and  prepare  for  EM 
vehicular  survey 

EM  61  Mk  II  calibration  tests, 
hardware  failure,  equipment 
shipped  to  Canada  for  repair 

11,  12  Sep 

Begin  vehicle  mag  survey  of 

South  and  Seed  Target  Areas 

Survey  200  X  600  m  area, 
including  the  seed  target 
area 

1 1  data  files,  9.6  survey  hours 

12-14  Sep 

Begin  vehicle  mag  survey  of  north 
area 

Survey  325  X  400  m  area 

13  data  files,  10.0  survey  hours 

14,  15  Sep 

Begin  vehicle  mag  survey  of  west 
area 

Survey  525  X  325  m  area 

14  data  files,  11.3  survey  hours 

17  Sep 

Vehicle  mag  survey  analysis, 

South,  seed.  North,  and  West  areas 

Completed  target  analysis 
and  prepared  spreadsheets 

17  Sep 

Deploy  EM61  Mkl 

Center  sensor  failed 
calibration 

Shipped  sensor  to  Canada  for 
repair 

18  Sep 

Deploy  EM61  Mk  I  on  Seed 

Target  Area 

Survey  without  center 
sensor 

3  data  files,  1.61  survey  hours 

19  Sep 

Deploy  EM61  Mk  II  following 
repairs 

Perform  calibration  tests 

20  Sep 

Deploy  EM61  Mk  II  on  Seed 

Target  Area 

Complete  N/S  survey 

1 1  data  files,  6.3  survey  hours, 
repaired  EM61,  Mk  I  received 

21  Sep 

Deploy  EM61  Mk  II  on  Seed 

Target  Area 

Complete  E/W  survey 

14  data  files,  6.8  survey  hours 

22  Sep 

Deploy  EM61  Mk  I  on  Seed 

Target  Area 

Complete  E/W  survey 

8  data  files,  6.0  survey  hours 

22  Sep 

Assemble  airborne  components 

Helo  arrives  on  site 

23  Sep 

Deploy  EM61  Mk  I  on  Seed 

Target  Area 

Complete  N/S  Survey 

8  data  files,  7.0  survey  hours 

23  Sep 

Install  platform  on  helo 

Conduct  practice  survey 
of  North  area 

1  data  file,  1.1  survey  hours, 

GPS  data  defective 

23  Sep 

Airborne  survey  of  South  and  seed 
areas 

1  data  file,  0.9  survey  hour 

24  Sep 

Deploy  EM61  Mk  I  on  Seed 

Target  Area  containing  60-  and 

81 -mm 

Survey  50  X  200  m  area, 
complete  E/W  survey 

3  data  files,  1.61  survey  hours 

24  Sep 

Airborne  platform  repairs 

Replace  GPS  antennas  and 
mag  sensor  5  and  cables 

25  Sep 

Airborne  surveys,  Airborne  South 
sorties 

Sortie  South  12 

Sortie  South  1 

Sortie  South  0 

Sortie  South  1 1 

Sortie  South  5 

Sortie  South  6 

16  acres,  0.4  survey  hours, 

107  acres,  1.5  survey  hours, 

107  acres,  2.0  survey  hours, 

26  acres,  0.5  survey  hours, 

88  acres,  1.1  survey  hours, 

78  acres,  0.9  survey  hours, 

68  acres,  0.9  survey  hours, 

34  acres,  0.6  survey  hours 

25  Sep 

Analysis  of  Airborne  South/seed 
data 

Completed  joint  target 
analysis  of  South  seed 
area 

Submitted  seed  dig  lists  to 

ESTCP  and  ERDC 

Table  3.  Activity  Log  for  the  Demonstration  Projects  on  the  IA.  (continued) 


Date 

Activity 

Result 

Comment 

26  Sep 

Airborne  surveys,  Airborne  South 
and  North  sorties 

Sortie  South  8, 

Sortie  South  9  (data  lost), 
Sortie  South  10 

Sortie  South  13 

Sortie  South  10,  reflight 
Sortie  North  1 

23  acres,  0.4  survey  hour 

47  acres,  0.7  survey  hour 

37  acres,  0.6  survey  hour 

5  acres,  0.1  survey  hour 

37  acres,  0.7  survey  hour 

93  acres,  1 .3  survey  hours 

87  acres,  1.5  survey  hours 

91  acres,  1.2  survey  hours 

47  acres,  0.7  survey  hour 

27  Sep 

Airborne  surveys,  reflights,  and 
calibrations 

South,  missed  area 
reflights 

Sortie  North  8 

Sortie  North  4 

Sortie  North  5,  Upper 
Plateau 

Sortie  North  6 

Sortie  North  5,  Lower 
Plateau 

Sortie  North 

0.5  survey  hour 

103  acres,  1.5  survey  hours 

96  acres,  1 .2  survey  hours 

44  acres,  0.5  survey  hour 

100  acres,  1.2  survey  hours 

55  acres,  0.7  survey  hour 

102  acres,  1.0  survey  hour 

0.3  hour 

27,  28  Sep 

Unmount  airborne  platform 

Helo  departs 

28  Sep 

Dig  teams  arrive  on  site 

Coordinate  with  tribal 
team  and  analysis  teams 

Practice  target  waypointing 

28  Sep 

Airborne  target  analysis 

Analysis  completed  and 
reconciled 

Airborne  spreadsheets  and 
dig  lists  prepared 

1  Oct 

Dig  teams  waypoint  South  and 
seed  areas 

Begin  recovering  seed 
targets 

?  Oct 

Target  recovery  from  ground 
surveys 

All  targets  dug  in  vehicle 
surveyed  areas 

Began  digging  airborne  targets 

19-23 

Nov 

Airborne  target  digging  terminated 

Final  blow-in-place 
demolition,  OE  scrap 
sorted  and  certified 

23  Nov 

Site  cleaned,  flags  removed 

Dig  teams  depart 

30  Nov 

All  logistics  support  removed 
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4.0  BADLANDS  BOMBING  RANGE  PERFORMANCE  ASSESSMENT 


4.1  PERFORMANCE  DATA 


4.1.1  Overview 


Initially,  the  vehicular  and  airborne  surveys  of  the  Seed  Target  Area,  shown  in  Figure  7,  were 
performed.  We  compared  the  relative  abilities  of  the  two  systems  to  detect  the  seeded  ordnance 
and  to  differentiate  between  clutter  and  ordnance.  Following  the  surveys  of  the  Seed  Target 
Area,  a  100-acre  area  was  jointly  surveyed  by  the  airborne  and  vehicular  systems.  In  Figure  7, 
the  100-acre  joint  vehicular  and  airborne  surveys  are  shown  as  three  separate  survey  areas:  the 
North,  South,  and  West  survey  blocks. 


The  remaining  areas  surveyed  by  the  airborne 
system  were  collectively  referred  to  as  the 
Airborne  Production  Survey  area.  The  overall 
performance  of  the  airborne  survey  system  was 
then  evaluated  and  compared  with  the  probable 
results  that  would  have  resulted  from  an 
independent  vehicular  survey. 

Figure  7  provides  a  perspective  of  the  vehicular 
surveys  conducted  during  this  demonstration 
compared  to  the  vehicular  surveys  conducted  in 
the  1999  demonstration.  The  areas  surveyed 
with  the  vehicular  system  in  the  2001 
demonstration  are  overlaid  in  blue  showing  their 
relationship  to  the  earlier  surveys. 

The  2001  vehicular  surveys  were  partitioned 
into  three  separate  areas  denoted  as  2001  West, 

2001  North,  and  2001  South.  The  2001  South 
block  is  contiguous  to  the  10-acre  Seed  Target 
Area.  The  Seed  Target  Area  was  extended  in 
this  way  to  create  the  Seed-South  survey  block 
because  the  longer  East-West  lanes  were  more 
efficient  to  survey  with  both  the  vehicular  and 
the  airborne  systems.  The  Seed  Target  Area  lies 
100-300  m  SE  from  the  center  of  the  bull’s  eye 
and  is  therefore  fairly  densely  populated  with 
shrapnel  and  clutter  left  behind  following  the 
1999  remediation.  The  remaining  20  acres  that 
constitute  the  2001  South  survey  block  lie  in  an 
area  that  had  not  been  previously  surveyed  by 

the  MTADS.  The  2001  Seed-South  block  was  independently  surveyed  using  the  Airborne 
MTADS  (Table  4)  to  provide  an  initial  data  set  for  comparative  analysis  with  the  vehicular  data. 
The  area  was  reflown  as  part  of  sortie  South  3  (see  Section  3.6.5  and  Figure  19  in  the  final 
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X(m) 


Figure  7.  Magnetic  Anomaly  Map  of  the  Areas 
Surveyed  in  1999.  (The  vehicular  survey  areas 
covered  in  the  200 1  demonstration  are  shown  in 
blue.) 
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report);  however,  the  data  used  for  target  analysis  of  the  2001  Seed-South  block  was  from  the 
initial  mission. 


Table  4.  Summary  of  the  Target  Reports  for  All  the  Vehicular  and  Airborne  Target 
Analyses  for  the  North,  West,  and  South  blocks  and  the  Seed  Target  Area. 


Survey 

Area 

Priority 

1 

Priority 

2 

Priority 

3 

Priority 

4 

Priority 

5 

Priority 

6 

Total 

Seed 

Target 

Area 

Vehicular  mag 

24 

15 

36 

3 

37 

55 

170 

Airborne  mag  analysis 

36 

34 

69 

14 

8 

24 

185 

V-mag  targets  not  in 
airborne  analysis 

- 

- 

- 

- 

- 

“ 

25 

South 

Survey 

Vehicular  mag 

6 

13 

25 

8 

17 

1 

70 

Airborne  Mag  Analysis 

5 

57 

80 

3 

6 

3 

154 

V-mag  targets  not  in 
airborne  analysis 

- 

“ 

- 

17 

West 

Survey 

Vehicular  mag 

13 

17 

45 

18 

43 

43 

179 

Airborne  mag  analysis 

43 

134 

129 

11 

23 

13 

353 

V-mag  targets  not  in 
airborne  analysis 

- 

- 

- 

- 

- 

- 

19 

North 

Survey 

Vehicular  mag 

2 

10 

11 

8 

10 

11 

52 

Airborne  mag  analysis 

16 

29 

24 

5 

12 

9 

95 

V-mag  targets  not  in 
airborne  analysis 

- 

- 

- 

- 

- 

- 

8 

Combined 

Totals 

Vehicular  mag 

45 

55 

117 

37 

107 

110 

471 

Airborne  mag  analysis 

101 

255 

303 

33 

49 

49 

790 

V-mag  targets  not  in 
airborne  analysis 

- 

- 

“ 

- 

- 

- 

69 

4.1.2  The  Seed  Target  Surveys 

The  Seed  Target  Area  is  pictorially  defined  in  Figure  7.  Figure  8  shows  magnetic  anomaly 
images  of  the  Seed  Target  Area  from  the  vehicular  and  airborne  surveys.  Many  of  the  inert 
ordnance  targets  are  apparent.  The  coordinate  system  in  these  images  (and  all  other  anomaly 
images  generated  by  the  MTADS  DAS)  is  a  user-defined  local  coordinate  system  in  meters.  The 
origin  of  the  local  coordinate  system  was  chosen  to  be  identical  to  that  used  in  the  1999  MTADS 
survey.  The  offset  between  local  and  Universal  Transverse  Mercator  (UTM)  coordinates  was 
recorded  at  the  top  of  all  target  analysis  spreadsheets. 

The  vehicular  analysis  was  performed  first.  The  airborne  analysis  was  carried  out  with  both  the 
vehicular  and  airborne  data  displayed  side  by  side,  as  shown  in  Figure  9.  In  the  Seed  Target 
Area,  the  airborne  target  picks  have  the  same  target  numbers  as  in  the  vehicular  survey  analysis. 
All  inert  buried  ordnance  was  detected  and  analyzed  in  both  the  vehicular  and  airborne  surveys. 
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Figure  8.  Magnetic  Anomaly  Images  of  the  Seed  Target  Area  from  the  Airborne  Survey  (left)  and  the  Vehicular 
Survey  (right).  (The  Seed  Target  Area  is  200  m  x  200  m;  the  southwest  comer  coordinates  are  X  =  360  m, 

Y  =  530  m.) 


Figure  9.  Magnetic  Anomaly  Maps  of  a  Portion  of  the  Seed  Target  Area  Presented  in  Pixel  Format. 

(The  airborne  survey  is  shown  on  the  left  and  the  vehicular  survey  on  the  right.) 
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4.1.3  The  South,  West,  and  North  Surveys 

4.1.3. 1  The  South,  West,  and  North  Vehicular  Surveys 

Figure  7  shows  the  relative  positions  of  the  2001  vehicular  magnetometer  survey  areas. 
Excluding  the  Seed  Target  Area,  the  remainder  of  the  vehicular  magnetometer  survey 
encompasses  99.6  acres  (40.3  hectares).  The  vehicular  data  were  analyzed  immediately  on  site 
in  preparation  for  the  anticipated  onslaught  of  airborne  data  once  the  Airborne  Production 
Survey  began.  The  data  were  processed  on  the  six-category  priority  scale. 

4.1.3.2  The  South,  West,  and  North  Airborne  Surveys 

The  Seed  Target  Area  and  the  South  block  were  initially  flown  on  September  23  as  a  single, 
continuous  200  m  x  600  m  mission  in  the  first  airborne  test  survey  at  the  IA.  Data  from  this 
initial  30-acre  survey  were  used  to  carry  out  the  airborne  target  analysis  of  the  Seed  Target  Area 
and  South  block. 

Target  digging  in  the  Seed  Target  Area  was  based  on  the  combined  analyses  of  the  vehicular 
magnetometer  and  EM  and  airborne  magnetometer  MTADS  survey  data.  Every  target  appearing 
in  any  of  the  dig  lists  was  dug.  Digging  in  the  South,  West,  and  North  blocks  was  based  on  only 
the  target  list  prepared  from  the  vehicular  magnetometer  array  survey.  The  airborne  data  were 
analyzed  retrospectively  for  the  South,  West,  and  North  blocks  because,  during  the  last  week  in 
September,  airborne  data  for  areas  of  the  IA  not  covered  in  the  1999  survey  or  in  the  2001 
vehicular  survey  were  being  analyzed  first  in  preparation  for  digging  targets  in  these  previously 
unsurveyed  areas.  The  airborne  and  vehicular  magnetometry  data  were  jointly  analyzed  for  the 
Seed  Target  Area,  as  described  in  Section  4.1.1  of  this  report  and  in  greater  detail  in  Section 
3.6.1  of  the  final  report,  to  develop  rules  for  the  airborne  analysis.  The  airborne  data  overlapping 
the  remaining  100  acres  of  the  vehicular  survey  were  analyzed  semi-independently  of  the 
vehicular  data.  This  means  that  the  airborne  target  anomalies  were  independently  chosen  and 
analyzed  using  only  the  airborne  data.  However,  the  results  were  carefully  scrutinized  by 
comparing  the  joint  data  sets  to  evaluate  the  rules  that  were  developed  during  the  Seed  Target 
Area  joint  analysis. 

4.1.4  Comparative  Performance  of  the  Two  Systems 

Table  5  summarizes  the  results  of  the  target  recovery  operations  on  these  survey  blocks  and  in 
the  Seed  Target  Area.  The  values  in  parentheses  refer  to  the  airborne  analyses.  All  inert  and  live 
UXO  detected  in  the  vehicular  survey  and  analyses  were  also  detected  in  the  airborne  survey. 
Interestingly,  target  121  in  the  Seed  Target  Area  that  was  incorrectly  classified  as  category  6,  OE 
scrap,  by  the  vehicular  survey  and  analysis  was  classified  as  a  category  3  UXO  target  in  the 
airborne  survey  and  analysis.  The  clutter  above  the  target,  which  confused  the  vehicular 
analysis,  was  not  an  interference  in  the  airborne  anomaly  signal. 
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Table  5.  Correlation  of  the  Ordnance  Recovery  Data  with  the  Analysis  Assignments  for 
the  Seed  Target  Area  and  the  North,  West,  and  South  Survey  Blocks. 


Survey  Area 

Category* 

Total 

1 

2 

3 

4 

5 

6 

Seed  Target 

Area 

Targets  analyzed 

24 

15 

36 

3 

37 

55 

170 

Targets  excavated 

24 

15 

36 

3 

37 

55 

170 

Inert  UXO 
recovered 

105-mm 

KISH! 

- 

- 

- 

- 

10 

155-mm 

raw 

Bin 

- 

- 

- 

1 

10 

8-in 

u jsm 

- 

- 

HOI 

- 

5 

Live  UXO 
recovered 

105-mm 

155-mm 

RM 

1 

1  | 

8-in 

South  Survey 

Targets  analyzed 

6 

13 

25 

8 

17 

1 

70 

Targets  excavated 

6 

13 

25 

8 

17 

1 

70 

Live  UXO 
recovered 

105-mm 

155-mm 

IIUH 

BOI 

- 

- 

- 

- 

2 

8-in 

Effin 

- 

- 

- 

- 

- 

2 

West  Survey 

Targets  analyzed 

13 

17 

45 

18 

43 

43 

179 

Targets  excavated 

13 

17 

45 

18 

43 

43 

179 

Live  UXO 
recovered 

105-mm 

155-mm 

ue» 

1 

- 

- 

- 

- 

2 

8-in 

OH 

- 

1 

- 

- 

- 

1 

North  Survey 

Targets  analyzed 

2 

10 

11 

8 

10 

11 

52 

Targets  excavated 

2 

10 

11 

8 

10 

11 

52 

Live  UXO 
recovered 

105-mm 

155-mm 

DOS 

- 

- 

- 

- 

- 

1 

8-in 

inn 

- 

- 

- 

- 

1 

Combined 

Totals 

Targets  analyzed 

45 

55 

117 

37 

107 

110 

471 

Targets  excavated 

45 

55 

117 

37 

107 

110 

471 

Live  UXO 
recovered 

105-mm 

155-mm 

EKH 

HI 

- 

- 

- 

- 

5 

8-in 

wmm. 

1 

- 

- 

- 

4 

Total  inerts 

19(21) 

ESI 

- 

- 

mn 

- 
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*  Values  in  parentheses  refer  to  recoveries  made  from  the  airborne  survey  and  analysis. 


Table  4  contains  a  summary  of  all  the  vehicular  and  airborne  target  analyses  comparing  the 
performances  of  the  two  systems  for  all  categories  of  targets.  The  most  striking  information  in 
Table  4  is  that  the  airborne  survey  analyses  contain  67%  more  targets  than  the  vehicular  surveys. 
This  is  the  result  of  the  effects  shown  in  Figures  8  and  9.  The  high-density  data  in  the  vehicular 
survey  enables  many  non-UXO  targets  to  be  excluded  from  consideration  on  the  basis  of  shape 
information  that  is  not  available  in  the  much  sparser  airborne  data.  The  airborne  analyses  also 
produce  priority  assignments  that  are  skewed  toward  the  priority  1,  2,  and  3  categories,  again 
because  the  shape  information  in  the  anomaly  signature  that  the  analyst  uses  is  not  present  to  the 
same  degree  in  the  airborne  data.  While  we  have  demonstrated  that  the  Impact  Area  can  be 
effectively  cleared  of  UXO  using  either  the  vehicular  or  the  airborne  survey  approaches,  the 
airborne  survey  necessarily  requires  more  targets  to  be  dug. 

In  the  following  discussion,  we  group  the  Seed  Target  Area  with  the  other  vehicular  surveys  and 
consider  both  inert  ordnance  and  live  UXO  as  ordnance.  On  this  110-acre  area,  of  the  471 
targets  analyzed  and  dug,  217  targets  had  been  classified  as  UXO  and  254  as  more  likely  not 
UXO.  We  recovered  35  intact  ordnance  targets.  In  the  vehicular  analysis,  24  ordnance  were 
classified  as  category  1,  seven  as  category  2,  one  as  category  3,  one  as  category  5,  and  one  as 
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category  6.  The  category  5  target  (an  inert  155-inm  projectile)  was  misclassified  because  it  had 
been  incorrectly  degaussed.  Target  121  (discussed  earlier  in  this  section)  was  also  misclassified 
as  category  6  in  the  vehicular  survey  analysis. 

Based  on  the  vehicular  analysis,  if  one  accepts  a  97%  (or  a  94%)  goal  for  the  UXO  cleanup 
process,  the  above  analysis  would  support  leaving  1 10  of  the  category  6  (or  217  of  the  categories 
5  and  6)  targets  undug  behind  the  vehicular  MTADS  survey.  Leaving  the  110  category  6  targets 
undug  would  leave  one  UXO  in  the  field.  Leaving  the  217  categories  5  and  6  targets  undug 
would  leave  two  UXO  in  the  field. 

A  final  observation  relating  to  the  target  analysis  process  should  be  made.  On  this  1 10-acre  site, 
47 1  anomalies  appear  in  the  target  spreadsheets.  On  the  basis  of  a  signal-intensity  threshold  or 
object-analyzed  size  threshold,  there  are  several  hundred  more  objects  in  the  area  that  would  be 
included  in  the  dig  list.  In  our  interactive  analysis,  these  additional  targets  were  excluded  either 
by  visual  inspection  of  the  anomaly  signature  or  by  trial  fits  of  the  anomalies.  These  additional 
objects  would  appear  in  the  dig  list  if  our  automated  target  picker  were  the  analyst  or  if  a  Mag 
and  Flag  team  did  the  survey. 

4,1.5  The  Airborne  Production  Survey 

Between  September  24  and  27  (Table  6),  all  easily  accessible  areas  of  the  IA  were  surveyed  with 
the  airborne  system.  The  IA  was  surveyed  in  an  east-west  direction.  The  operation  was  divided 
into  22  missions,  or  sorties  (see  Figure  10).  Over  most  of  the  area,  survey  lines  were  2.5-3  km 
long;  the  longer  sorties  were  designed  to  be  completed  in  about  an  hour.  Data  from  the  North 
1-3  and  South  0-8  airborne  (Figure  10)  sorties  were  extracted  to  conduct  separate  airborne 
analyses  of  the  vehicular  West  and  North  survey  areas. 

Table  6.  Airborne  MTADS  Survey  Production  Rates.  (Hours  in  parentheses  are  not  included 

in  survey  calculations.) 


Date 

Flight  Hours 

Survey  File 
Hours 

Survey  Acres 

Sunday  9/23 

Assembly 

C  alibration/training 

(2.4) 

(2.0) 

(66) 

Monday  9/24 

Survey 

5.1 

4.4 

313 

Tuesday  9/25 

Survey 

8.9 

7.9 

524 

Wednesday  9/26 

Survey 

8.6 

6.6 

383 

Thursday  9/27 

Survey 

7.5 

7.0 

465 

F  erry  time 

2.0 

Total 

32.1 

25.9 

1685 

Altogether,  about  250  acres  north  of  the  cross  fence  were  surveyed.  Most  of  this  area  is  50-75  ft 
lower  in  elevation  than  the  Bouquet  Table  and  was  not  likely  part  of  the  original  impact  area, 
although  overshoots  could  clearly  have  strayed  into  the  area. 
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A  coarse-scale  magnetic  anomaly  image  of  the 
entire  airborne  survey  area  is  shown  in  Figure 
11.  At  this  scale,  the  fence  lines,  geological 
features,  the  MTADS  support  trailers,  and  (in  a 
few  cases)  individual  buried  targets  are  visible. 
The  entire  site  was  divided  into  five  separate 
survey  blocks  because  of  its  size  and  because 
two  or  three  people  were  working  on  target 
analysis  at  the  same  time.  Intermittently,  the 
analysts  reviewed  each  other’s  outputs  to 
ensure  consistency.  On  Friday  September  28, 
the  target  analysis  was  completed  and  reviewed 
for  consistency.  On  Saturday  September  29, 
the  target  dig  sheets  were  prepared  from  the 
spreadsheets,  and  the  fdes  were  prepared  for 
loading  into  the  GPS  way  pointing  equipment. 
A  calibration  way  point  target  was  set  up  and 
flagged  east  of  the  equipment  trailer.  This 
target  appeared  at  the  top  of  each  list  of  targets 
to  be  acquired  and  flagged  each  day  before 
work  began  in  the  field. 


Figure  10.  Layout  for  the  Individual  Sorties 
Flown  by  the  Airborne  MTADS  Surveying  the 
Impact  Area. 


The  airborne  targets  in  Table  7  were  dug  by  analysis  category.  All  82  category  1  and  176 
category  2  targets  were  dug.  Only  270  of  486  category  3  targets  were  recovered;  216  category  3, 
and  449  category  4-6  targets  remain  undug.  Two  dig  teams  worked  independently  each  team 
with  two  EOTI  explosives-certified  members  and  one  OST  explosives  technician.  Each  team 
worked  with  hand  tools  and  backhoes,  depending  on  the  size  and  depth  of  the  individual  target 
being  prosecuted.  All  metallic  objects  associated  with  each  flag  were  recovered  and 
photographed,  and  the  hole  was  cleared  using  a  metal  detector  before  closing. 

4.2  PERFORMANCE  CRITERIA 


Performance  evaluation  criteria  include  system  detection  sensitivity,  accuracy  of  location  and 
depth  predictions,  and  navigation  system  performance. 

4.3  PERFORMANCE  ASSESSMENT 


Our  program  performance  objective  was  to  test  the  operation  of  the  Airborne  MTADS  in  a 
realistic  survey  against  the  performance  of  the  vehicular  system  and  against  other  competing 
technologies  including  Mag  and  Flag.  The  objective  of  the  1999  Mag  and  Flag  clearance 
conducted  by  Air  Force  EOD  teams  was  to  flag  targets  larger  than  3  in  at  depths  less  than  1.5  ft. 
Based  on  the  1999  clearance  reports,  the  Mag  and  Flag  clearance  of  this  range  did  not  effectively 
lead  to  the  discovery  or  removal  of  the  live  high  explosive  (HE)-filled  dud  projectiles;  only  one 
live  projectile  was  found  in  the  1999  clearance.  Much  of  the  IA  is  significantly  contaminated 
with  small  metallic  clutter,  OE  shrapnel,  fencing  material,  and  auto  body  parts.  This  problem  is 
so  pervasive  that  it  effectively  defeats  the  use  of  nonrecording  sensors.  Using  the  handheld 
sensors  typically  employed  in  Mag  and  Flag  surveys,  it  is  very  difficult  to  differentiate  target 
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Y(m) 


size.  Setting  the  sensor  sensitivity  to  detect  a  105-mm  projectile  at  1.5  ft  will  ensure  that  it  rings 
off  on  a  2-in  to  3-in  piece  of  shrapnel  near  the  surface.  On  much  of  this  range,  the  correctly 
tuned  sensor  would  constantly  alarm,  leading  to  thousands  of  flags  per  acre.  It  is  not  clear  from 
the  1999  clearance  reports  whether  the  Mag  and  Flag  sensors  were  calibrated  against  projectiles 
buried  at  the  required  detection  limits. 
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Figure  11.  Magnetic  Anomaly  Image  for  the  Airborne  MTADS  Survey  of  the  Impact  Area. 
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Table  7.  Summary  of  the  Target  Analysis  and  Recovery  Operations  Following  the 

Airborne  Survey. 


Survey  Area 

Category 

Total 

1 

2 

3 

4 

5 

6 

Targets  Analyzed 

82 

176 

486 

208 

155 

86 

1,193 

Targets  Excavated 

82 

176 

270 

- 

- 

- 

528 

Live  UXO 
Recovered 

105-mm 

- 

- 

- 

- 

- 

- 

- 

155-mm 

- 

3 

3 

- 

- 

- 

6 

8-in 

- 

2 

2 

- 

- 

- 

4 

The  data  collection  approach  used  by  the  vehicular  magnetometer  MTADS  is  appropriate  for 
making  the  classification  decisions  that  enable  confidently  leaving  ~  90%  of  the  metallic  scrap 
items  in  the  field.  The  results  of  the  1999  MTADS  survey  demonstrated  that  this  can  be 
accomplished,  and  the  analysis  of  the  Seed  Target  Area  in  this  demonstration  verified  it.  The 
relatively  high  ratio  of  OE  scrap  recoveries  to  live  projectile  recoveries  from  the  vehicular 
magnetometer  survey  was  again  driven  by  the  curious  fact  that  unintended  postimpact 
detonations  of  the  155-mm  and  8-in  projectiles  leave  shrapnel  cluster  patterns  that  often  cannot 
be  distinguished  from  intact  105-mm  projectiles. 

4.3.1  Detection  Sensitivity 

The  ability  to  detect  targets  depends  primarily  on  the  sensitivity  of  the  sensors,  the  completeness 
of  the  survey  coverage,  and  the  signal-to-noise  ratio  in  the  survey  data.  The  Airborne  MTADS 
detection  efficiency  in  the  Seed  Target  Area  and  in  the  100-acre  common  area  survey  for  the 
105-mm,  155-mm,  and  8-in  projectiles  was  indistinguishable  from  that  of  the  vehicular  survey. 
Each  detected  all  the  seed  area  UXO  (inert  and  live)  and  detected  the  same  UXO  projectiles  in 
the  100-acre  common  survey  area.  The  airborne  system  can  clearly  be  relied  on  to  detect  buried 
projectiles.  It  should  also  be  noted  that  among  the  airborne  target  digs  was  a  2.75-in  inert  rocket 
warhead  that  had  evidently  wandered  away  from  whatever  its  mission  was  supposed  to  be. 

The  lower  density  of  the  airborne  data 
made  classification  decisions  more 
difficult  compared  with  the  vehicular 
data.  On  the  110  acres  surveyed  by  both 
systems,  60%  more  targets  would  have 
to  be  dug  behind  the  airborne  survey 
than  behind  the  vehicular  MTADS  if  all 
targets  (category  1-6)  were  dug.  The 
cost  implications  of  this  effect  are 
discussed  in  Section  4.4.2,  Cost 
Perfonnance,  and  Section  4.4.3,  Cost 
Comparison  of  this  report.  If  digging 
were  limited  to  category  1-5  targets, 
approximately  half  as  many  targets 


Figure  12.  ROC  Curves  for  the  Vehicular  and  Airborne 
Surveys  on  the  110-Acre  Common  Survey  Area. 
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would  have  to  be  dug  behind  the  vehicular  survey.  Another  way  of  visualizing  this  information 
is  with  a  plot  called  the  receiver  operating  characteristic  (ROC).  Figure  12  shows  a  comparison 
of  the  ROC  curves  for  the  1 10-acre  surveys  common  to  the  vehicular  and  airborne  systems. 

4.3.2  Missed  Targets 

Table  8  shows  data  relating  to  the  inert  targets  in  the  Seed  Target  Area.  This  information  enables 
comparisons  of  the  performance  of  the  airborne  and  vehicular  arrays.  Each  survey  approach  led 
to  selection  and  analysis  of  all  the  inert  targets.  Target  254  (an  8-in  projectile)  has  also  been 
included,  although  it  was  buried  just  north  of  the  Seed  Target  Area.  The  live  155-mm  projectile 
(target  104)  is  also  included.  The  vehicular  magnetometer  array,  because  it  produces  very  high- 
density  sensor  data,  generates  more  accurate  location  predictions.  An  average  deviation  from  the 
ground  truth  of  15  cm  is  typical. 

The  sensor  data  from  the  airborne  system  has  about  one  tenth  the  density  of  the  vehicular  data, 
but  this  data  density  is  still  sufficient  to  provide  location  accuracies  of  better  than  25  cm.  The 
ground  truth  data,  compiled  by  ERDC,  provide  the  depth  to  the  shallowest  part  of  the  buried 
item.  The  MTADS  DAS  predicts  the  depth  to  the  center  of  the  target.  With  the  exception  of 
target  86  (large  remnant  moment)  and  target  121  (overlaid  by  shrapnel),  all  ordnance  items  were 
categorized  as  1,  2,  or  3. 
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Table  8.  Vehicular  and  Airborne  Survey  Comparisons  with  the  Ground  Truth  in  the  Seed 

Target  Area. 
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Table  8.  Vehicular  and  Airborne  Survey  Comparisons  with  the  Ground  Truth  in  the  Seed 

Target  Area,  (continued) 


MTADS 

ID 

Survey 

UTM  X(m) 

UTM  Y(m) 

I  XY  (m) 
Vehicular 
Mag 

IXY(ra) 

Airborne 

Mag 

Depth 

(m) 

Size 

(m) 

Moment 

In  cl 

Azi 

Fit 

Quality 

Analyst  Comments 

uxo 

Categor 

y 

G3S-142 

Vehicular  Mag 

722,931.84 

4,838,164.64 

0.02 

0.58 

0.101 

0.5826 

34 

30 

0.954 

105mm 

l 

Airborne  Mag 

722931.77 

4838164.94 

0.29 

0.00 

0.093 

0.4570 

30 

23 

0.927 

105mm 

2 

Ground  T ruth 

722931.82 

4838164.65 

0.25 

105  mm 

0 

30 

Flat 

G3S-148 

Vehicular  Mag 

722,886.96 

4,838,168.65 

0.06 

0.79 

0.156 

2.1836 

63 

188 

0.986 

155mm 

1 

Airborne  Mag 

722886.83 

4838168.91 

0.25 

0.25 

0.152 

2.0118 

84 

262 

0.988 

1 55mm 

1 

Ground  T ruth 

722886.90 

4838168.67 

0.30 

155  mm 

35 

360 

Nose  Down 

G3S-149 

Vehicular  Mag 

722,879.12 

4,838,174.30 

0.15 

0.67 

0.138 

1.5182 

51 

312 

0.982 

105/1 55mm,  E/W,  nose  down 

1 

Airborne  Mag 

722879.38 

4838174.22 

0.15 

0.26 

0.138 

1.4937 

59 

307 

0.989 

155mm  E/W 

1 

Ground  T ruth 

722879.23 

4838174.21 

0.40 

105  mm 

'  45 

115 

Nose  Down 

G3S-152 

Vehicular  Mag 

722,831.29 

4,838,176.14 

0.16 

1.33 

0.146 

1.7633 

82 

90 

0.939 

possible  deep  155 

2 

Airborne  Mag 

722831.82 

4838176.25 

0.40 

0.46 

0.119 

0.9520 

74 

96 

0.965 

105 

1 

Ground  T ruth 

722831.42 

4838176.23 

0.92 

105  mm 

75 

5 

G3S-153 

Vehicular  Mag 

722,824.52 

4,838,171.35 

0.22 

1.21 

0.148 

1.8602 

90 

356 

0.970 

probable  deep  155 

1 

Airborne  Mag 

722824.43 

4838171.26 

0.32 

0.52 

0.137 

1.4690 

84 

195 

0.968 

155mm 

1 

Ground  T ruth 

722824.74 

4838171.34 

0.75 

8  in 

75 

350 

Nose  Down 

G3S-154 

Vehicular  Mag 

722,813.23 

4,838,176.30 

0.05 

1.36 

0.235 

7.4160 

87 

3 

0.941 

deep  8-in,  nose  down 

1 

Airborne  Mag 

722813.49 

4838176.36 

0.22 

0.55 

0.193 

4.1193 

89 

147 

0.988 

8  in 

1 

Ground  T ruth 

722813.27 

4838176.31 

0.60 

155  mm 

80 

15 

Nose  Down 

G3S-163 

Vehicular  Mag 

722,853.25 

4,838,196.21 

0.17 

0.65 

0.135 

1.4049 

47 

299 

0.972 

155mm,  E/W 

1 

Airborne  Mag 

722853.20 

4838196.21 

0.22 

0.37 

0.149 

1.8740 

43 

301 

0.960 

155mm  E/W 

1 

Ground  T ruth 

722853.42 

4838196.19 

0.25 

105  mm 

35 

1 10 

Nose  Down 

G3S-167 

Vehicular  Mag 

722,921.08 

4,838,201.86 

0.29 

1.01 

0.108 

0.7287 

64 

359 

0.921 

possible  deep  105mm 

2 

Airborne  Mag 

722921.44 

4838202.25 

0.25 

0.42 

0.106 

0.6742 

40 

38 

0.895 

possible  105 

3 

Ground  T ruth 

722921.32 

4838202.03 

0.60 

155  mm 

40 

30 

Nose  Down 

254 

Vehicular  Mag 

722792.22 

4838243.01 

0.08 

1.40 

0.233 

7.2576 

79 

189 

0.973 

great  8-in  signature 

1 

Airborne  Mag 

722792.37 

4838243.13 

0.15 

0.213 

5.73206 

79 

100 

0.977 

8 -in 

1 

Ground  T ruth 

722792.23 

4838243.09 

0.65 

8-in 

285 

75 

Nose  Down 

4.4  TECHNOLOGY  COMPARISON 
4.4.1  Technical  Performance 

In  this  section,  we  present  a  comparison  of  the  technical  performance  of  the  vehicular  and 
Airborne  MTADS  platforms  with  respect  to  system  reliability,  speed,  ease  of  use,  etc.  This 
demonstration  was  the  first  use  of  the  Airborne  MTADS  to  conduct  an  extensive  survey.  On  the 
Impact  Area,  the  airborne  hardware  performed  flawlessly  in  the  field,  and  the  data  processing, 
analysis,  and  target  picking  performance  was  routine,  exceeding  our  expected  production  rates. 


The  vehicular  magnetometer  array  has  been  deployed  on  a  dozen  large  sites.  Its  performance  is 
reliable  and  predictable,  partly  the  result  of  system  design,  but  more  importantly,  it  is  the  result 
of  careful  and  extensive  attention  to  maintaining  a  comprehensive  inventory  of  system  spares 
and  our  ability  to  effectively  recover  from  breakdowns  in  the  field  by  making  innovative 
decisions  and  tackling  mechanical,  hardware,  or  software  fixes  on  the  fly.  We  have  used  this 
same  resilience  in  design  and  the  redundancy  in  spares  with  the  airborne  system.  It  is 
worthwhile  to  note  that  in  the  field,  on  the  first  day  of  airborne  surveying,  we  built  new 
mounting  fixtures,  installed  new  GPS  antennas,  and  changed  out  one  of  the  magnetometers  (with 
its  interfaces  and  cable  runs)  with  parts  from  our  spares  inventory. 


The  production  rates  of  the  airborne  system  were  300-500  survey  acres  per  day  under  the 
conditions  of  this  site.  The  corresponding  production  rates  for  the  vehicular  MTADS  are 
routinely  18-24  acres  per  day.  This  ratio  of  production  rates  of  a  factor  of  15  in  favor  of  the 
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airborne  system  will  likely  hold  across  a  wide  range  of  site  conditions.  The  production  rates 
with  the  vehicular  system  would  be  much  lower  if  terrain  conditions  were  significantly  more 
difficult.  Production  rates  with  the  airborne  system  will  significantly  suffer  only  if  the  sites 
chosen  for  its  use  are  very  small  or  if  very  short  flight  lines  must  be  flown  with  difficult  turn¬ 
arounds. 

4.4.2  Cost  Performance 

Table  9  presents  a  cost  breakdown  for  a  hypothetical  1,500-acre  survey  on  a  relatively  benign 
site  typified  by  the  IA  at  the  BBR.  We  assume  that  the  site  is  a  UXO  range,  that  we  have  to 
establish  navigation  control  points,  that  the  site  would  not  benefit  from  a  preliminary  surface 
sweep/clearance,  that  a  1,000-mile  ferry  of  equipment  is  required,  that  we  have  to  provide  all 
logistics  support,  that  data  will  be  processed  and  analyzed  on  site  and  that  a  target  list  will  be 
prepared,  that  we  are  not  supporting  any  target  remediation,  and  that  a  report  (typical  of  an 
ESTCP  demonstration  report)  will  be  retrospectively  written.  During  the  survey  operation,  our 
daily,  on-site  costs  are  ~  $15,000.  For  the  purposes  of  scaling  the  size  of  the  survey  (probably  up 
to  3,000  or  4,000  acres),  one  should  be  able  to  assume  400  acres/day  of  survey  at  the  nominal 
daily  costs.  A  safe  projection  should  be  ~  $50  per  additional  acre. 

Table  9.  Projected  Costs  for  a  1,500-Acre  Airborne  MTADS  Survey. 


Preparation  and  Startup 

Site  Operations 
(Assume  1-Week  Operation) 

Mobilization/Demobilization 

Activity 

$K 

Activity 

$K 

Activity 

$K 

Site  visit  and 
inspection 

4 

Three  rental  vehicles 

2 

Rental  truck 

5 

Preparation  of  test 
plan,  maps,  photos, 
etc. 

15 

Supervisor**(1260+160)X7 

10 

Rental  truck  driver  (6  travel 
days)  (850+160)X6 

6 

Establish  control 
points 

6 

Helicopter/back  seat 
(850+160)X7 

7.5 

Helo  ferry  cost 

12 

Capital  equipment* 

- 

Analysis  support  (two 
persons)  (850+160)X7X2 

15 

Helo  ferry  pilot  (800+160)X2 
(2+0. 5X8)2 

2 

Permitting  and 

regulatory 

requirements 

Pilot  (5  days)  (800+160)X5 

5 

Three  workers’  travel  (assume 
one  in  the  truck) 

10 

On-site  logistics 

Charter  (2  days  setup,  tear 
down,  calibration/training) 

6 

Analysis 

15 

Office  trailer 
electrician,  power, 
fuel 

12 

Charter  (3  days  survey), 

(2+0. 5X8)3 

18 

Report 

25 

Security 

3 

Helo  fuel  truck/fuel 
(2800+300X2) 

Materials 

2 

Portable  toilets 

0.5 

Subtotal 

42.5 

66.9 

75 

*  MTADS  equipment  is  not  expensed  or  amortized  for  this  exercise. 

**  Personnel  costs  include  per  diem;  7-day  operation  assumed  unloading,  setup,  cleanup,  etc. 
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4.4.3  Cost  Comparison 


Excluding  the  cost  of  the  report,  the  cost  of  the  hypothetical  1,500-acre  airborne  survey  projected 
in  Table  9  is  $  106/acre.  In  this  section,  we  consider  the  relative  costs  of  a  vehicular  MTADS 
survey  and  remediation  compared  to  using  the  airborne  system.  Assume  the  same  1,500  acre 
range  as  included  in  the  airborne  survey  described  in  Table  9.  The  hypothetical  vehicular  survey 
uses  one  fewer  support  person  than  we  actually  used  at  the  IA,  but  the  daily  salary  and  per  diem 
costs  are  ~  20%  higher  than  were  paid  in  2001  at  the  IA,  in  line  with  those  used  in  the  airborne 
calculation,  reflecting  current  rates. 

In  most  surveys  with  the  vehicular  MTADS,  covering  20  acres  per  day  in  hospitable  areas  is 
routine.  Because  this  is  a  very  extended  survey  for  a  single  vehicular  system,  we  assume  that  the 
weekends  must  be  reserved  for  maintenance  and  repair  and  to  make  up  for  weather  delays. 
Therefore,  surveying  1,500  acres  with  the  vehicular  MTADS  is  projected  to  require  ~  75  days  or 
16  five-day  weeks.  Assuming  1 -month  personnel  rotations  for  the  supervisor,  the  vehicle  driver, 
and  the  two-man  analysis  trailer  crew,  and  assuming  that  this  staff  is  supported  by  three  OST 
members,  we  made  the  cost  projections  shown  in  Table  10.  The  projected  survey  costs  are 
$581,000,  or  $390/acre. 

Table  10.  Projected  Costs  for  a  1,500-Acre  Vehicular  MTADS  Survey. 


Preparation  and  Startup 

Site  Operations  (Assume  16-week 
operation) 

Mobilization  &  Demobilization 

Activity 

$K 

Activity 

$K 

Activity 

$K 

Site  visit/inspection 

4 

Three  rental  vehicles  (4x4  for  16 
wk) 

32 

Rental  trailer  truck 

10 

Preparation  of  test 
plan,  maps,  photos, 
etc. 

15 

Supervisor** 

([1260/day]5+[160/day]7)xl6wk 

119 

Rental  truck  driver  (6 
travel  days) 

(850+160)X6 

6 

Establish  control 
points 

6 

Analysis  support  (two  persons) 
([850/day]5+[160/day]7xl6wk) 
two  persons 

172 

Five  airfare  round-trips 
x  four  rotations 

20 

Capital  equipment* 

- 

Driver/support 

([850/day]5+[160/day]7xl6wk) 

86 

Truck  driver,  one  air 
round-trip 

1 

Permitting  and 

regulatory 

requirements 

Fuel,  vehicle  repair,  maintenance 

6 

Equipment  repair, 
restock 

10 

On-site  logistics 

OST  support 

32 

Office  trailer, 
electrician,  power, 
fuel 

25 

Report 

15 

Security 

- 

Materials 

5 

Portable  toilets 

4 

Tent  cover 

3 

Tribal  subcontracting 

10 

Subtotal 

72 

447 

62 

*MTADS  equipment  is  not  expensed  or  amortized  for  this  exercise 

**  Personnel  costs  include  per  diem,  80-day  operation  assumed  unloading,  setup,  cleanup,  etc. 
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During  the  remediation  of  the  IA  in  2001,  we  dug  471  vehicular  targets  and  527  airborne  targets 
for  a  total  of  998.  Remediation  operations,  including  way  pointing,  digging,  blowing,  sorting, 
certifying,  and  disposal  of  scrap,  cost  $200,000.  Equipment  rental  costs  were  $20,000;  the  GPS 
equipment,  already  on  site,  was  considered  rent  free.  Target  recovery  costs  were  therefore 
$220/target.  This  is  in  line  with  our  typical  costs  of  ~  $200/target. 

On  the  IA,  1,565  acres  were  surveyed,  including  the  110-acre  vehicular  survey  area  and  the 
remaining  1,455  acres  of  the  airborne  survey  area  that  were  analyzed  and  remediated  based  on 
the  airborne  survey.  A  total  of  1,938  targets  (1,193  +  790)  were  specified  from  the  airborne 
analysis  (in  all  priority  categories).  The  ratio  of  vehicular  to  airborne  targets  picked  on  the 
vehicular  survey  areas  was  471/790.  On  this  basis,  we  project  that  there  would  be  1,182  targets 
(471/790  x  1,983)  to  remediate  if  the  vehicular  MTADS  were  used  to  survey  all  1,565  acres. 
This  information  is  summarized  in  Table  11.  Without  the  requirement  to  extensively  document 
the  dig  sheets  and  maintain  a  digital  photographic  log  of  all  dug  targets,  we  estimate  that  targets 
could  be  dug  at  a  cost  of  $200  per  target. 

The  predicted  total  survey  and  clearance  costs  of  $563,000  for  the  airborne  operation  are  realistic 
because  they  closely  reflect  actual  survey,  analysis,  and  remediation  costs.  The  vehicular  survey 
and  analysis  values  are  more  hypothetical.  We  have  never  undertaken  a  vehicular  operation  of 
this  magnitude.  It  was  costed  based  on  our  use  of  engineers  and  Ph.D.s  to  staff  the  analysis 
trailer  and  supervise  field  crews.  In  a  realistic  commercial  vehicular  survey,  manpower  costs 
would  be  lower  on  a  dollar/hour  basis.  It  is  conjectural  as  to  whether  the  20-acres-per-day 
survey  rates  could  be  attained  (or  maintained)  with  less  qualified,  less  motivated  crews. 

Table  11.  Hypothetical  Survey  and  Remediation  Costs  (in  $K)  for  a  1,565-Acre  Survey  on 

the  BBR  Impact  Area.  (Primary  cost  entries  assume  all  targets  are  dug.  Costs  in  parentheses 

assume  that  only  category  1-5  targets  are  dug.) 


Airborne  Clearance 

$K 

Vehicular  Clearance 

$K 

Projected  survey  cost 

166.4 

Projected  survey  cost 

581.0 

Projected  cost  to  clear  1,983 
targets 

396.6 

(371.1) 

Projected  cost  to  clear  1,182 
targets 

236.4 

(180.6) 

Total  airborne  survey  and 
remediation  costs 

563.0 

(526.8) 

Total  vehicular  survey  and 
remediation  cost 

817.4 

(761.6) 
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5.0  ABERDEEN  PROVING  GROUND  DEMONSTRATION  DESIGN 

5.1  PERFORMANCE  OBJECTIVES 

The  objectives  of  this  demonstration  were  established  and  defined  by  APG  in  their  Wide  Area 
UXO  Aerial  Demonstration  and  Survey  Project  as  documented  in  their  demonstration  test 
plan.20  Multiple  sites  at  APG  were  established  to  evaluate  the  performance  of  the  NRL  Airborne 
MTADS  in  comparison  with  the  U.S.  Army  Corps  of  Engineers  (USACE)/Huntsville-ORNL 
airborne  system  (ESTCP  Projects  UX-0037  and  UX-0101).  The  APG  demonstration  test  plan20 
specified  that  each  system  would  fly  the  same  survey  areas  during  the  same  demonstration 
period.  Survey  products  from  both  the  NRL  and  ORNL  surveys  were  to  be  submitted  to  the 
AEC,  ESTCP,  and  the  Institute  for  Defense  Analyses  (IDA)  for  evaluation.  Five  survey  ranges 
were  prepared,  in  addition  to  a  small  calibration  area  with  known  UXO  challenges.  To  augment 
existing  UXO  and  clutter,  which  was  present  on  four  of  the  five  survey  areas,  the  U.S.  Army 
Aberdeen  Test  Center  (ATC)  emplaced  additional  inert  seed  UXO  on  three  of  the  survey  areas, 
ranging  in  size  from  60-min  mortars  to  155-mm  projectiles.  Specific  objectives  “  included 
demonstrating  the  following: 

•  The  detection  capability  on  a  relatively  low-clutter  area  seeded  with  small  and  medium¬ 
sized  UXO 

•  The  detection  and  discrimination  capabilities  on  a  mixed-use  range  with  relatively  flat 
terrain  and  low  vegetation  levels 

•  The  detection  and  discrimination  capabilities  on  a  very  complex,  mixed-use  range  with 
areas  of  2-m  high  vegetation,  transitions  to  shallow  water,  high  levels  of  surface  clutter 
and  obstacles,  and  expectations  of  buried  UXO  caches 

•  The  UXO  detection  capability  in  freshwater  ponds  seeded  with  ordnance 

•  The  UXO  detection  capability  on  a  marine  projectile  impact  area  with  water  depths  of 
0-2.5  m. 


Perfonnance  goals  were  conducting  efficient  airborne  surveys;  analysis  of  data,  including 
differentiation  of  UXO  from  clutter;  and  preparation  of  data  products,  including  target  reports 
and  ranked  analysis  results. 

5.2  PERFORMANCE  METRICS 

5.2.1  Target  Location  Accuracy 

The  surveys  were  conducted  with  Airborne  MTADS  using  GPS  navigation;  dual  GPS  antennas; 
and  acoustic,  laser,  and  radar  altimeters.  Three  of  the  sites  were  seeded  with  inert  ordnance,  and 
the  ground  truth  for  the  site  was  held  by  ATC  until  after  the  analysis  results  had  been  submitted. 

5.2.2  System  Operational  Performance 

Field  logs,  kept  for  all  survey  activities,  provided  infonnation  about  setup  times,  survey  times, 
production  rates,  and  equipment  performance.  The  electronic  data  files  provided  additional 
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information  about  field  survey  performance  by  documenting  survey  data  collection  times,  the 
course  used  in  data  collection,  stoppages  during  data  collection,  how  turnarounds  were 
accomplished,  and  how  precisely  the  pilot  completed  the  planned  survey  grid.  Missed  areas 
were  detennined  and  difficulties  with  navigational  data  were  documented,  and,  where  possible, 
corrected. 

5.2.3  Detection  Capability 

This  demonstration  further  tested  the  Airborne  MTADS’  ability  to  differentiate  between  clutter 
and  ordnance  on  land  and  was  the  first  use  of  the  airborne  system  to  conduct  UXO  surveys  over 
water.  Because  the  sensors  on  an  airborne  platform  must  be  deployed  farther  from  the  ground 
surface  than  those  on  vehicular  or  man-portable  systems,  it  is  understood  that  detection 
sensitivity  for  single,  smaller  UXO  items  is  compromised.  In  an  airborne  survey,  the  sensors  are 
4  to  20  times  more  distant  from  the  ground  surface  than  in  a  vehicular  survey.  The  primary 
effects  of  this  are  a  significantly  decreased  peak  signature  intensity  and  a  substantial  spreading  of 
the  dimensions  of  the  anomaly  signature  in  the  airborne  data. 

5.3  SELECTING  TEST  SITES 

The  criteria  and  requirements  leading  to  the  choice  of  test  sites  for  this  demonstration  are 
explained  in  the  APG  demonstration  test  plan.  In  general,  the  site  managers  selected  areas  that 
had  different  UXO  challenges  (ranging  from  antipersonnel  submunitions  to  large  general 
purpose  (GP)  bombs),  various  densities  of  targets  and  clutter,  different  types  of  terrain,  and 
varying  difficulties  of  access  (vegetation,  water,  stockpiled  munitions,  heavy  machinery,  etc.). 
The  individual  survey  areas  were  small  by  airborne  survey  standards,  varying  from  much  less 
than  an  acre  to  slightly  over  100  acres.  Therefore,  there  was  no  significant  possibility  of 
evaluating  the  economies  of  the  airborne  system  compared  to  land-based  approaches. 

5.4  TEST  SITE  HISTORY  AND  CHARACTERISTICS 

A  description  of  the  impact  ranges  and  the  prepared  test  sites  at  APG  is  provided  in  the 
demonstration  test  plan”  prepared  by  APG.  Information  pertinent  to  our  specific  operations  is 
briefly  reviewed  in  Sections  4. 6. 2.2,  4. 6.2. 3,  4. 6. 2.4,  and  4. 6. 2. 5  of  the  plan. 

5.5  PHYSICAL  SETUP  AND  OPERATIONS 
5.5.1  Site  Preparation 

The  only  site  preparation  work  carried  out  specifically  in  preparation  for  these  demonstration 
surveys  was  the  burying  of  seed  and  calibration  targets  at  the  Airfield,  and  seed  targets  at  Active 
Recovery  Field,  the  Chesapeake  Bay  Impact  Area,  and  the  dewatering  ponds.  The  seed  targets 
were  not  buried  at  the  water  sites  but  were  placed  flush  with  the  bottom  sediments. 

The  five  test  sites  chosen  by  APG  comprise  parts  of  four  current  or  former  impact  ranges  and  a 
prepared  site  at  the  Airfield.  At  three  of  the  sites — the  Airfield,  Mine,  Grenade,  and  Direct-Fire 
Range,  and  offshore  areas  of  the  Bay  Impact  Area — selected  target  areas  were  seeded  by  ATC 
with  inert  ordnance.  Seed  targets  specified  in  the  APG  demonstration  test  plan  20  included 
60-mm  and  81-inm  mortars,  2.75-in  rocket  warheads,  and  105-tnm  and  155-inm  projectiles. 
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5.5.2  Changes  in  the  MTADS 


The  changes  in  the  MTADS  following  the  Badlands  Bombing  Range  demonstration  included  the 
implementation  of  software  routines  to  produce  a  digital  elevation  model  (DEM)  using  the 
MTADS  altimeters  and  the  modifications  in  the  DAS  to  present  the  analyst  with  a  real-time 
depth  fit  for  analyzed  targets.  In  addition,  a  utility  was  created  to  save  the  selected  data  clips 
used  for  target  analysis  and  the  values  for  the  maximum  positive  and  negative  signal  intensities 
for  fit  targets.  This  information  was  requested  by  the  Program  Office  specifically  for  the  APG 
demonstration. 

5.6  ANALYTICAL  PROCEDURES 

Survey  data  were  inspected  on  site,  at  offices  provided  in  the  pilot’s  ready  lounge  at  the  Airfield, 
using  notebook  computers  running  the  Windows  version  of  the  Airborne  MTADS  DAS. 
Separate  project  files  were  established  for  each  survey  and  individual  sortie  files  were  integrated 
into  single  master  files  for  each  of  the  survey  projects.  The  only  areas  resurveyed  during  the 
demonstration  were  the  calibration  and  seed  target  sites  at  the  Airfield.  The  initial  data  taken  at 
these  sites  were  primarily  used  for  pilot  orientation  and  equipment  checkout  and  were  not  used  in 
target  analysis. 

Each  data  file  was  edited  to  remove  data  from  aircraft  turnarounds  and  from  well  outside  the 
survey  boundaries.  Sensor  data  were  inspected  and  spurious  data  points  were  edited  from  the 
files.  A  500-point  (5-second)  demedian  filter  was  applied  separately  to  each  sensor  track.  This 
suppressed  zero-offset  differences  among  the  sensors,  long-term  sensor  drift,  heading  offsets, 
and  large-scale  geology  effects.  A  notch  filter  (at  6.45  Hz  and  12.9  Hz)  was  applied  to  suppress 
blade-  (rotor  hub)  induced  noise  and  (at  25  Hz)  to  suppress  platform  vibration  noise.  The  notch- 
filter  widths  and  roll-offs  were  adjusted  and  applied  equally  to  all  sensors.  Values  were  chosen 
to  null  blade  noise  from  the  two  outboard  sensors  at  each  end  of  the  array.  The  three  center 
sensors,  which  were  closer  to  the  blade  footprint,  retained  minimal  blade-based  noise  at  a  level 
that  did  not  interfere  with  analysis  of  the  smallest  (60  mm)  expected  targets.  All  data  processing 
and  target  analysis  took  place  subsequent  to  the  end  of  the  fieldwork.  Each  data  set  was 
processed  using  the  same  approach  and  evaluation  parameters  by  a  single  analyst  who  also 
prepared  all  dig  lists  and  report  products. 
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6.0  ABERDEEN  PROVING  GROUND  PERFORMANCE  ASSESSMENT 


6.1  PERFORMANCE  DATA 

6.1.1  The  Airfield  Survey 

Target  analysis  was  carried  out  using  the  MTADS  DAS.  Raw  data  were  processed  as  described 
in  Section  4.6.8. 1  of  the  final  report,  Airborne  UXO  Surveys  Using  the  MTADS.  The  initial 
target  analysis  assumed  that  the  smallest  targets  of  interest  were  60-mm  mortars  and  the  largest 
were  155-mm  projectiles.  As  the  survey  image  in  Figure  13  shows,  there  are  many  magnetic 
anomalies  on  this  site  that  are  significantly  larger  than  155-mm  projectiles.  Buried  utilities, 
likely  conduits  for  runway  landing  lights,  lie  roughly  parallel  to  the  east,  south,  and  west  survey 
boundaries.  On  the  south,  the  utility  run  lies  beyond  the  limit  of  the  survey.  However,  both  the 
east  and  west  boundaries  of  the  survey  include  the  utility  runs.  Many  of  the  larger  signals 
associated  with  these  features  are  unlikely  to  involve  UXO;  however,  in  the  northwest  corner  of 
the  site,  there  is  a  significantly  disturbed  area  in  which  the  aerial  photo  and  the  DEM  both  show 
features  that  resemble  craters.  The  magnetic  anomaly  map  shows  that  significant  magnetic 
signatures  are  associated  with  many  of  these  features.  In  addition,  there  are  a  few  dozen  isolated 
substantial  target  returns  within  the  survey  area  that  could  be  large  UXO.  Therefore,  our 
analysis  reports  both  targets  in  the  seed  target  size  range  and  others  that  are  too  large  to  be 
155-mm  projectiles. 
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Figure  13.  Pixel  Image  Plot  (subsampled)  of  the  Airborne  MTADS  Survey  of  the  Airfield.  (The 
white  border  defines  the  limits  of  the  survey.  See  Figure  25  in  the  final  report  for  the  digital  orthoquad 

[DOQ]  and  DEM  presentations.) 
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The  target  list  includes  both  small  and  large  targets.  The  probability  that  an  individual  target  in 
this  list  is  one  of  the  seed  targets  is  ranked  using  the  six-category  subjective  analysis  criteria 
established  during  the  Jefferson  Proving  Ground  demonstrations.  All  large  targets  in  the  survey 
area  are  included  in  the  target  report,  even  though  many  are  clearly  too  large  to  be  members  of 
the  class  of  seed  targets.  The  column  in  the  target  report  labeled  “Probability  as  UXO  Seed” 
evaluates  the  data  on  the  basis  of  there  being  only  five  ordnance  types  of  interest  on  the  site.  A 
probability  of  5  or  6  for  a  very  large  target  indicates  a  very  low  probability  of  that  object  being  a 
seed  target;  the  probability  of  that  object  being  a  UXO  larger  than  the  class  of  seed  targets  may 
be  significantly  greater. 

The  first  318  targets  in  the  Airfield  target  report  are  those  included  in  the  initial  submission 
based  upon  60-mm  mortars  being  the  smallest  UXO  of  interest  on  the  site.  The  data  were 
reanalyzed,  as  directed  by  the  Program  Office,  to  pick  targets  down  to  the  system  or  site  noise 
limit  following  reprocessing  of  the  data  as  described  above.  Targets  319-618  resulted  from  the 
follow-up  analysis. 

6.1.2  System  Performance  at  the  Airfield 

The  IDA  analysis  and  report  of  the  demonstration  performance  at  APG  summarizes  the  site 
information  and  detection  perfonnance.  Table  12  shows  that  52  inert  UXO,  primarily  81-inm 
mortars  and  105-mm  projectiles,  were  seeded  into  the  prepared  range.  IDA  considered  the 
effects  of  using  1.0  m,  1.5  m,  and  2.0  m  halos  on  detection  performance  and  94%  of  the 
MTADS’  correct  declarations  were  captured  in  the  1.0  m  halo.  Figure  14  presents  this 
information  (for  a  1.5  m  detection  halo)  in  a  ROC  curve  format.  All  target  declarations  were 
made  using  the  six-category  probability  scale.  These  probability  bins  were  used  to  construct  the 
ROC  curve.  Overall,  the  MTADS  correctly  identified  slightly  more  than  94%  of  the  UXO. 

Table  12.  Ordnance  Detection  Results  for  the  Airfield  Open  Area  for  Three  Detection 

Halos.* 


Ordnance 

Emplaced 

lm 

1.5m 

2m 

60-mm 

3 

3 

3 

3 

81 -mm 

21 

16 

18 

18 

105-mm 

28 

27 

28 

28 

Total 

52 

46 

49 

49 

*  Adapted  from  Table  5  of  Reference  21 


6.1.3  The  Active  Recovery  Field  Survey 

The  survey  of  the  Active  Recovery  Field  covered 
~  100  acres.  The  magnetic  anomaly  image  is  shown 
in  Figure  15.  This  highly  contaminated  site  (see 
Figures  3 1  and  32  in  the  final  report,  Airborne  UXO 
Surveys  Using  the  MTADS)  is  characterized  by 
clusters  of  large  and  small  ordnance;  stockpiles  of 
recovered  ordnance  and  scrap;  an  extremely  dense 
ordnance  deposit  stretching  for  more  than  200  m 
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Figure  14.  ROC  Curves  for  the  Airfield 
Open-Field  Area  for  a  1.5  m  Halo. 


and  lying  offshore  in  the  bay  parallel  to  the 
shoreline;  areas  of  dense,  6-ft-tall  vegetation;  and 
by  several  steel  blast  shields  and  scattered  heavy 
equipment.  It  was  within  this  context  of  signal 
returns  many  times  larger  than  a  signal  generated  by 
a  155-inm  projectile  that  data  analysis  was  carried 
out.  Where  background  levels  allowed,  targets 
were  analyzed  to  the  size  level  that  would  include 
60-mm  mortars.  The  target  analysis  of  this  survey 
required  >100  hours  of  analysis  time.  The  IDA 
report  discloses  that  64  seed  targets  were  buried 
amidst  the  clutter  at  the  Active  Recovery  Field. 
Tables  13  and  14  summarize  the  detection  results 
for  the  airborne  surveys.  NRL  declared  2,969 
targets..  The  detection  efficiency  at  this  site  was 
only  marginally  above  random  chance. 

6.1.4  The  Dewatering  Ponds 
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The  entire  survey  area  at  the  dewatering  ponds 
consisted  of  five  freshwater  ponds.  Figure  16 
shows  a  survey  plot  of  the  four  small  ponds  called 
the  Finger  Ponds  by  APG.  The  image  extends  north 
and  south,  well  beyond  the  ends  of  the  ponds. 

There  is  a  small  missed  survey  area  near  the  center 
of  the  south  end  of  the  westernmost  pond  and  a 
small  missed  area  (due  to  data  dropout)  on  the  western  edge  of  the  second  pond  from  the  east. 


Figure  15.  Magnetic  Anomaly  Image 
(interpolated)  of  the  Active  Recovery  Field. 

(Note  the  cluster  of  surface  ordnance  at  the  top 
center,  stockpiles  of  materials  along  the  road, 
and  the  extended  concentration  of  magnetic 
returns  offshore.) 


Table  13.  Ordnance  Detection  Results  for  Active  Recovery  Field  for  Two  Detection  Halos.* 


Ordnance 

Emplaced 

1  m 

1.5  m 

81 -mm 

32 

0 

1 

105-mm 

32 

4 

4 

Total 

64 

4 

5 

*  Adapted  from  Table  8  of  Reference  21 


Table  14.  Cumulative  Detection  Probability  as  Function  of  Ordnance  Likelihood  Call  for 

Active  Recovery  Field.* 


UXO  Likelihood 

%  detections 

1-m  halo 

1.5-m  halo 

1 

3.1 

4.7 

2 

4.7 

6.3 

3 

4.7 

6.3 

4 

6.3 

7.8 

5 

6.3 

7.8 

6 

6.3 

7.8 

*  Adapted  from  Table  9  of  Reference  21 
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Figure  17  shows  a  magnetic  anomaly  image 
from  the  survey  of  the  large  pond  at  the 
eastern  edge  of  the  area.  Most  of  the  more 
intense  signals  are  from  objects  lying  at  or 
beyond  the  hanks  of  the  pond.  A  much  finer 
scale  is  required  to  visualize  the  UXO  lying 
on  the  bottom  of  the  pond. 

1 

> 

Only  about  130  of  the  224  declared  targets 
at  the  dewatering  ponds  are  small  enough  to 
be  seed  targets,  and  many  of  these  lie 
outside  the  pond  areas.  The  larger  targets 
and  the  targets  beyond  the  pond  shorelines 
are  included  in  the  target  report  because,  in 
the  APG  Demonstration  Test  Plan,  this 
survey  area  was  claimed  to  be  relatively  free 
of  clutter.  This  target  information  is 
provided  so  that  the  targets  can  be 
investigated  if  there  is  an  interest  in  their 
identities.  Table  15  shows  the  results  from 
the  IDA  analysis  and  report.  Forty-seven 
targets,  mostly  81 -mm  mortars  and  105-mm 
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Figure  17.  Magnetic  Anomaly  (subsampled,  pixel) 
Image  from  the  Survey  of  the  Large  Dewatering 
Pond. 


Figure  16.  Pixel  Image  Plot  of  the  Survey  of 
the  Finger  Ponds. 

projectiles,  were  placed  in  the  ponds.  NRL’s 
analysis  declared  224  targets,  as  described 
above.  At  the  dewatering  ponds,  the  primary 
difficulty  in  identifying  the  targets  resulted  from 
the  standoff  distance  between  the  targets  and  the 
sensors  (the  intervening  water  and  air)  rather 
than  the  background  clutter,  which  interfered 
with  detection  at  the  Active  Recovery  Field. 

Table  16  shows  the  ground  truth  coordinates  for 
the  seed  targets  emplaced  in  the  five  dewatering 
ponds.  The  center  column,  offset  by  double 
lines  on  the  left  and  right,  provides  comments 
generated  when  the  ground  truth  data  was 
rationalized  with  the  survey  images. 

The  combined  standoff  distance  of  the 
helicopter  above  the  water  surface  and  the  depth 
of  the  water  above  the  seed  targets  rendered  all 
of  the  81 -mm  targets  undetectable.  Effectively, 
all  of  the  105-mm  and  155-mm  targets  were 
detected  in  the  small  ponds.  One  target  (FP- 
105MM  2)  was  missed  because  its  easting 
coordinate  had  been  recorded  incorrectly  in  the 
target  report. 
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Table  15.  Cumulative  Detection  Probability  as  a  Function  of  Ordnance  Likelihood  Call  for 

the  Pond  Surveys.* 


UXO  Likelihood 

%  detections 

1  -m  halo 

1.5-m  halo 

1 

19.1 

19.1 

2 

25.5 

29.8 

3 

27.7 

31.9 

4 

27.7 

31.9 

5 

27.7 

31.9 

6 

27.7 

31.9 

*  Adapted  from  Table  7  of  Reference  21. 


The  ground  truth  for  the  dewatering  ponds  seed  targets  was  provided  by  Mr.  Gary  Rowe.  Its 
release  was  delayed  to  allow  demonstration  tests  of  other  systems  on  the  ponds.  The  coordinates 
and  identifications  of  the  seed  targets  are  provided  in  Table  16,  which  also  provides  information 
on  the  targets  that  were  detected  in  the  MTADS  survey.  We  reexamined  the  signatures  of  the 
targets  that  were  not  detected,  and  in  the  center  column  of  the  table,  we  provide  our  observations. 

6.1.5  The  Mine,  Grenades,  and  Direct-Fire  Range 

The  Range,  shown  in  Figure  18,  was  the  largest  of  the  survey  areas  at  130  acres.  A  north-south 
paved  road  that  is  apparent  in  the  magnetic  anomaly  image  bisects  the  survey.  To  the  west  of  the 
road  is  a  series  of  gravel  roads  leading  to  target  pads.  During  the  Airborne  MTADS  survey,  the 
pads  were  not  occupied.  The  bluestone  used  to  construct  the  gravel  roads  and  pads  is  very 
magnetically  active.  Figure  19  shows  part  of  the  upper  road  and  the  target  pad.  The  individual 
target  anomalies,  ranging  in  size  from  fuzes  and  antipersonnel  ordnance  to  GP  bombs,  are 
generally  clustered  about  the  target  pads.  The  large  amount  of  missed  area  along  the  south¬ 
eastern  side  of  the  survey  was  the  result  of  the  tree  cover.  The  easternmost  tip  of  the  survey  is 
dominated  by  high  signal  returns.  Much  of  this  area,  as  observed  during  the  survey,  is 
characterized  by  construction  rubble  from  previously  existing  structures. 

Seed  targets  were  not  placed  in  this  area.  Therefore,  the  analysis  was  carried  out  assuming  that 
the  survey  was  in  preparation  for  cleanup  of  a  mixed-use  range.  The  target  report  contains 
almost  3,400  targets.  There  are  eight  areas  that  we  considered  to  be  too  densely  cluttered  to 
successfully  analyze.  If  these  areas  are  designated  for  clearance,  they  should  be  surface  cleared 
and  then  surveyed  using  either  the  man-portable  or  the  vehicular  MTADS  magnetometer  arrays. 
The  much  higher  density  data  would  enable  targets  to  be  analyzed  more  accurately.  Much  of  the 
remainder  of  the  survey  area  could  be  effectively  remediated  {not  cleared)  using  the  airborne 
survey  and  analysis. 
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Table  16.  Ground  Truth  for  the  Targets  Emplaced  in  the  Five  Dewatering  Ponds. 


Ground  Truth 

MTADS  Assignment 

Location 

Target 

Number 

Serial 

No. 

Azi. 

Depth 

(m) 

Rationalize  Ground  Truth  with  Survey  Data 

MTADS 
Target  ID 

Depth 
DEM  (m) 

Size  (m) 

Moment 

Fit  Quality 

Analyst  Comments 

Large 

Pond 

P-81  MM  1 

172 

0 

H 

targ  114  is  1.5m  East,  overlaid  with  too  many 
high  passes 

P-81  MM  2 

131 

90 

■n 

not  picked,  3nT  signal,  lost  in  noise 

P-81  MM  3 

127 

45 

no  signal 

P-81  MM  4 

170 

0 

lost  to  signal  from  huge  targ  78 

P-81  MM  5 

129 

45 

not  picked,  4nT  signal  in  3nT  noise 

P-81  MM  6 

100 

45 

no  signal 

P-81  MM  7 

174 

90 

no  signal 

P-81  MM  8 

133 

90 

no  signal 

P-81  MM  9 

132 

0 

signal  lost  to  targ  14  &  15 

P-81  MM  10 

20 

0 

no  signal 

P-81  MM  11 

139 

90 

no  signal 

P-81  MM  12 

173 

0 

no  signal 

P-105MM  1 

195 

0 

lost  under  target  115 

P-105MM  2 

178 

90 

247 

1.45 

0.73 

105mm 

P-105MM  3 

210 

45 

target  in  missed  area 

P-105MM  4 

200 

0 

no  signal 

P-105MM  5 

189 

0 

no  signal 

P-105MM  6 

207 

45 

no  signal 

P-105MM  7 

162 

45 

2.1 

no  signal 

P-105MM  8 

197 

0 

target  246 

P-105MM  9 

161 

45 

target  243,  2  m  South  because  it  was  2  targets 

243 

3.53 

0.147 

1.7324 

0.49 

155mm 

P-105MM  10 

145 

45 

0.59 

0.62 

105mm 

P-105MM  11 

186 

90 

0.95 

0.69 

105mm 

P-105MM  12 

172 

90 

1  think  targ  245  moved  by  1 .5  m 

245 

4.28 

0.59 

medium  target,  deep 

P-105MM  13 

138 

0 

mm 

no  signal 

P-105MM  14 

159 

90 

m 

lost  in  huge  negative  anomaly 

P-105MM  15 

174 

0 

lost  in  noise 

P-105MM  16 

179 

45 

P-105MM  17 

221 

45 

lost  in  noise 

P-105MM  18 

134 

90 

2.1 

lost  in  noise 

P-155MM  1 

111 

0 

■n 

target  79,  likely  moved  ~  1m 

79 

1.89 

0.70 

155mm  at  6  ft 

P-155MM  2 

104 

45 

no  signal,  target  moved? 

P-155MM  3 

105 

90 

target  14 

14 

1.75 

0.67 

below 

P-155MM  4 

Lost 

90 

mm 

lost  in  target  78  signal 

Small 

FP-81MM  1 

169 

45 

surrounded  by  203,  204,  205,  not  picked 

Ponds 

FP-81MM  2 

123 

45 

not  picked,  4nT  signal  in  2nT  noise 

FP-81MM  3 

136 

90 

lost  under  target  1 64 

FP-81MM  4 

180 

0 

lost  under  target  154,  155 

FP-105MM  1 

141 

0 

target  191,  too  big  for  105mm  ? 

191 

1.79 

0.72 

difficult  fit,  155mm 

FP-105MM  2 

147 

0 

0.3 

target  189,  my  coordinate  may  be  wrong  in 
table 

189 

* 

part  signature,  wont  fit 

FP-105MM  3 

140 

90 

MU 

187 

0.19 

0.89 

105mm/2.75in 

FP-105MM  4 

198 

45 

target  202 

202 

0.17 

0.83 

105mm 

FP-105MM  5 

193 

45 

target  166 

166 

0.35 

0.93 

105/1 55mm 

FP-105MM  6 

171 

0 

target  168,  shadowed  by  167 

167 

1.21 

0.88 

large  deep  taqrget 

FP-105MM  7 

177 

90 

target  1 52 

152 

0.00 

0.70 

shallow  target,  60/81  mm 

FP-105MM  8 

185 

90 

target  1 53 

153 

0.38 

0.73 

shallow,  81mm 

FP-155MM  1 

106 

45 

target  186 

186 

1.64 

0.86 

large  target  at  6  ft 

FP-155MM  2 

109 

0 

target  164 

164 

0.41 

0.96 

large  shallow  target,  155mm 

To  undertake  a  comprehensive  UXO  clearance  of  this  range  would  require  several  clearances 
and  resurveys.  There  is  a  substantial  amount  of  small  ordnance  and  aluminum  ordnance  visible 
on  the  surface.  The  final  survey,  therefore,  should  be  done  with  an  EM  array,  which  would 
likely  defeat  the  high  magnetometer  return  from  the  bluestone  pads  and  roads.  From  an 
economic  point  of  view,  if  this  area  were  designated  for  clearance,  it  would  be  more  economical 
to  start  over.  One  should  first  conduct  a  surface  clearance,  repeat  the  magnetometer  survey,  dig 
targets,  then  survey  with  an  EM  array  and  dig  targets  again. 
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Figure  18.  MTADS  Survey  Image  of  the  Mine, 
Grenades,  and  Direct-Fire  Range. 


6.1.6  The  Chesapeake  Bay  Impact  Range 

An  interpolated  magnetic  anomaly  image  of 
the  Chesapeake  Bay  survey  is  shown  in  Figure 

20.  The  area  surveyed  was  well  offshore 
because  we  lost  signal  from  the  GPS  base 
station  and  there  was  not  another  station 
available  within  line-of-sight  for  the 
helicopter  to  continue  surveying  closer  to 
shore.  The  covered  survey  area  included 
about  30  acres  that  provided  a  good  estimation 
of  the  target  density  in  the  area. 

The  offshore  target  report  contains  800 
targets.  The  targets  are  much  denser  at  the 
northeast  end  of  the  survey,  although  the 
entire  survey  area,  as  exemplified  by  Figure 

21,  reflects  an  impact  area.  Because  of  the 
significant  standoff  distance  between  the 
targets  and  the  sensor  boom,  the  target 
signatures  spread  and  tended  to  overlap. 
Water  depths  were  uncertain  but  were  likely 
in  the  range  of  2.5-6  ft.  From  the  shape  of  the 
anomaly  signatures  and  the  analyzed  target 


Figure  19.  Magnetic  Anomaly  Image  of  a  Portion 
of  the  Range  Showing  the  Target  Pad  Near  the 
North  Corner  of  the  Survey  in  Figure  18. 
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Figure  20.  Magnetic  Anomaly  Image  (interpolated) 
of  the  Chesapeake  Bay  Impact  Range  Survey. 
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depths,  the  water  is  probably  shallower  near 
the  north  end  of  the  survey.  The  average 
analyzed  target  sizes  are  much  larger  than  the 
105-mm  projectiles  that  were  cited  in  the  APG 
test  plan  as  the  likely  dominant  UXO. 

Because  of  the  relatively  large  separation 
between  the  sensors  and  the  targets  buried  in 
the  sediment,  larger  targets  are  more  visible  in 
our  analysis,  and  in  some  cases,  multiple 
targets  may  make  contributions  to  individual 
target  fits.  It  is  our  estimation  that  many,  if 
not  most  of  the  targets  in  the  target  report  are 
very  large  projectiles  or  GP  bombs.  This 
would  be  an  ideal  area  for  conducting  an 
underwater  survey  with  the  marine  MTADS 
system.  Comparison  of  the  data  sets  would  be 
very  instructive. 

6.2  PERFORMANCE  CRITERIA 

These  demonstration  surveys  were  intended  to 
evaluate  the  performance  of  the  Airborne 
MTADS  in  a  series  of  relatively  small  surveys 
at  ordnance  ranges  and  impact  areas  with 
various  types,  sizes,  and  densities  of  ordnance 
and  OE  (and  non-OE)  clutter.  Performance  goals  as  stated  in  the  original  test  plan  were  based  on 
detection  of  inert  seed  targets  at  the  Airfield,  the  dewatering  ponds,  the  Active  Recovery  Field, 
and  the  offshore  range.  IDA  personnel  evaluated  the  results  of  the  data  analyses  submitted  by 
the  demonstrators;  see  Section  4.7  of  Airborne  UXO  Surveys  Using  the  MTADS.  The  portion  of 
the  offshore  range  that  was  surveyed  did  not  contain  seed  targets,  nor  did  the  Mine,  Grenade,  and 
Direct-Fire  Range.  The  detection  performance  discussion  below  is  based  on  information 
provided  by  IDA  in  their  analyses  of  the  target  reports.  IDA  evaluated  the  relative  detection 
efficiencies  and  location  accuracies  and  biases  of  the  two  airborne  systems. 

In  addition  to  evaluating  the  ability  of  the  airborne  systems  to  detect  UXO  in  a  variety  of 
settings,  the  demonstrations  were  intended  to  evaluate  the  relative  survey  production  efficiencies 
of  the  platforms,  the  system  deployment  strategies  and  efficiencies,  the  data  processing  and 
preparation  approaches,  the  target  analysis  efficiency  and  accuracy,  the  ability  of  the  systems  to 
distinguish  between  intact  UXO  and  clutter,  the  relative  abilities  of  the  demonstrators  to  create 
survey  products  that  could  support  a  realistic  UXO  remediation,  and  the  ability  to  create 
geographically  based  and  GIS-compatible  survey  products.  Also,  survey  costs  and  cost 
efficiencies  were  to  be  evaluated  and  compared.  The  larger  surveys  on  the  offshore  range  and 
the  Mine,  Grenade,  and  Direct-Fire  Range,  in  addition  to  providing  information  to  APG  about 
relative  contamination  levels  on  the  ranges,  enabled  system  evaluation  against  some  of  the 
objectives  that  were  not  specifically  target-related. 
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Figure  21.  Pixel  Image  (subsampled)  of  an 
Area  Near  the  South  End  of  the  Offshore 
Range  Showing  Individual  Target 
Signatures. 
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The  performance  information  from  the  IDA  reports,  addressed  in  Section  4.7  of  Airborne  UXO 
Surveys  Using  the  MTADS,  was  presented  effectively  without  comment.  In  the  following 
sections,  we  readdress  each  of  the  sites  that  had  seed  targets,  then  discuss  the  additional  targets 
that  were  dug  at  Active  Recovery  Range  from  lists  prepared  by  IDA  using  the  MTADS  and  Oak 
Ridge  Airborne  Geophysical  System  (ORAGS)  target  reports.  Perfonnance  evaluation  criteria 
include  those  described  above  and  system  detection  sensitivity,  accuracy  of  location  and  depth 
predictions,  and  navigation  system  performance. 

6.3  PERFORMANCE  ASSESSMENT 

6.3.1  Performance  at  the  Airfield 

In  Section  4. 6. 8.2  of  Airborne  UXO  Surveys  Using  the  MTADS,  we  reported  that,  at  the  request 
of  the  ESTCP  Program  Office,  we  extended  the  analysis  of  the  airfield  data.  The  original 
analysis  assumed  that  the  smallest  UXO  of  interest  were  60-nnn  mortars.  In  the  reanalysis,  we 
were  directed  to  report  all  targets  down  to  the  size  limit  (signal-to-noise  limit)  of  detection.  Our 
original  target  report  contained  308  targets;  the  expanded  analysis  contained  610  targets.  In 
Section  4.7.1  of  Airborne  UXO  Surveys  Using  the  MTADS,  we  presented  tabular  data  and  ROC 
curves  prepared  by  IDA  showing  the  MTADS’  detection  and  characterization  performance.  IDA 
analyses  were  based  on  our  expanded  list  containing  610  targets. 

In  the  original  analysis  containing  308  targets,  two  of  the  three  60-mm  and  fourteen  of  twenty- 
one  81-mm  mortars  were  correctly  reported.  The  105-mm  projectiles  were  all  detected;  one  of 
the  projectiles  (NRL  Target  No.  248,  StringID  PAF-105MM  1A)  was  reported  10  cm  beyond  the 
1.0-m  detection  halo. 

In  the  expanded  analysis  (610  total  reported  targets),  the  final  60-nnn  mortar  was  reported,  as 
were  four  additional  81-mm  mortars.  This  left  three  81-mm  mortars  undeclared.  In  each  case 
(NRL  Target  Nos.  572,  191,  and  259),  declarations  were  recorded;  however,  the  signatures  of  the 
larger  objects  masked  those  of  the  81-mm  seed  targets,  causing  a  seed-target  miss  in  each  case. 

The  original  analysis,  which  involved  308  targets,  captured  44  of  the  52  (or  85%)  of  the  seed 
targets,  including  all  the  105-mm  projectiles.  The  false-alarm  rate  for  this  analysis  was  then  six 
digs  for  each  recovered  seed  target.  The  302  additional  targets  in  the  expanded  report  captured  5 
additional  seed  targets.  Only  one  of  the  five  was  a  target  with  a  fit  that  converged.  The  four 
remaining  targets  were  unanalyzable  items  mechanically  marked  in  dense  clutter  consisting 
primarily  of  large  targets.  Digging  these  targets  might  recover  the  additional  five  seed  targets; 
however,  it  is  debatable  whether  the  analysis  really  isolated  these  seed  targets.  Unless  they  are 
specifically  instructed  to  “dig  the  flag,”  EOD  personnel,  digging  targets  in  the  field  typically 
orient  themselves  with  a  metal  detector  to  begin  their  operation.  If  the  dig  team  felt  their  mission 
was  to  dig  the  large  target  (either  specified  by  our  dig  list  or  with  guidance  from  their  metal 
detector),  once  they  recovered  the  large  target,  they  may  or  may  not,  recover  the  nearby  smaller 
seed  target. 

Digging  all  targets  in  the  expanded  target  report  would  lead  to  a  false-alarm  rate  of  1 1.5  digs  per 
recovered  seed  target.  If  all  targets  were  dug,  the  final  probability  of  detection  (Pd)  would  be 
94%,  and  three  81-mm  projectiles  would  be  left  in  the  field.  At  this  point,  it  is  a  matter  of 
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conjecture  whether  the  originally  submitted  Airfield  dig  list  or  the  expanded  dig  list  represents 
the  better  survey  work  product. 

The  detection  efficiency  of  the  Airborne  MTADS  at  the  Airfield  (using  either  dig  list)  was 
exceptionally  high.  The  missed  targets  on  the  expanded  dig  list  were  undetectable  because  they 
were  buried  in  the  footprint  of  much  larger  targets.  These  results,  on  their  face,  would  indicate 
that  the  MTADS  could  be  used  to  detect  60-mm  and  81 -mm  mortars.  The  Airfield  was  an 
unusual  situation,  however;  the  results  cannot  be  extrapolated  to  other  sites.  This  site  is  very  flat, 
the  background  clutter  density  (except  in  the  area  of  the  buried  utilities)  is  relatively  low,  and 
geological  interference  is  nonexistent.  There  is  no  significant  vegetation  on  the  site — It  looks 
much  like  a  golf  course  fairway.  For  these  reasons,  we  were  able  to  fly  the  survey  at  an 
unusually  low  altitude,  and,  because  the  site  is  very  small,  we  also  flew  it  very  slowly.  As  was 
pointed  out  in  the  IDA  report,  our  data  density  at  the  APG  Airfield  was  about  three  times  higher 
than  is  typical  of  many  of  our  larger  airborne  surveys. 

6.3.2  The  Active  Recovery  Field 

As  described  in  Section  4.7.2  of  Airborne  UXO  Surveys  Using  the  MTADS,  the  seed  target 
detection  efficiency  at  the  Active  Recovery  Site  was  vanishingly  small.  The  evaluation  provided 
in  Table  13,  which  shows  five  correctly  declared  targets  within  a  1.5  m  radius,  is  misleading. 
Examination  of  the  target  analyses  for  these  five  targets  shows  that  three  of  the  five  NRL 
declarations  were  accidental,  resulting  from  analyzed  objects  that  were  much  too  large  to  be  the 
implanted  seed  targets.  The  Active  Recovery  Field  survey  area  is  much  too  contaminated  with 
very  large  ferrous  objects  to  enable  detection  of  the  seed  targets.  The  massive  signatures  of  the 
very  large  objects  effectively  screen  the  returns  from  the  much  smaller  seed  targets.  The  density 
of  large  targets  and  their  overlapping  signatures  require  that  target  analysis  be  done  on  a  much 
less  sensitive  scale  than  on  any  of  the  other  sites  in  this  demonstration. 

Conducting  UXO  surveys  on  a  site  with  the  conditions  of  Active  Recovery  Field  is  a  waste  of 
resources.  UXO  geophysical  surveys  should  be  conducted  only  following  removal  of  heavy 
equipment,  hardware,  and  movable  obstacles  such  as  the  packing  crates  and  blast  screens. 
Moreover,  a  preliminary  surface  clearance  should  always  be  conducted  on  a  site  as  contaminated 
as  this  one.  Even  assuming  that  these  steps  have  been  taken,  geophysical  surveys  (if  UXO 
clearance  is  the  goal)  on  a  site  such  as  this  will  always  have  to  be  done  several  times.  In  each 
survey  and  clearance  cycle,  efforts  should  be  concentrated  only  on  the  largest  targets  in  dense 
areas;  more  sparsely  contaminated  areas  can  be  more  comprehensively  cleaned  in  each  cycle.  To 
confidently  clear  an  area  like  this  range  would  require  several  sequential  survey  and  clearance 
operations  (at  least  three,  following  the  initial  surface  clearance). 

We  declared  ~  3,000  targets  in  the  Active  Recovery  Field  dig  report.  Conducting  the  target 
analysis  for  this  site  using  the  MTADS  routines  and  preparing  the  target  report  and  required 
graphics  products  were  very  time-consuming;  the  realistic  cost  was  >$12,000.  This  far  exceeds 
the  original  survey  cost  for  the  site.  Searching  for  the  seeded  81 -mm  and  105-mm  targets  on  this 
range,  without  first  removing  the  existing  contamination,  was  shown  to  be  an  effectively 
impossible  task. 
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To  increase  the  value  of  the  Active  Recovery  Field  study,  IDA  worked  with  personnel  from 
APG,  ATC,  and  ESTCP  to  develop  a  selective  dig  list  of  additional  (pre-existing)  targets  for 
remediation.  The  MTADS  and  ORAGS  target  reports  were  sorted  to  establish  common  target 
picks.  These  were  down-selected  to  targets  that  were  relatively  isolated  from  interferences  and 
to  targets  assigned  relatively  high  UXO  probabilities.  The  dig  list  prepared  by  IDA  contained 
291  targets.  The  ATC  dig  list  was  pared  to  218  targets  during  the  process  of  digging.  Of  the 
targets  in  the  ATC  list,  29  were  not  dug  because  they  were  offshore  (or  for  other  placement 
reasons),  or  the  results  were  lost  or  were  inconclusive.  The  final  dig  report  is  presented  in  the 
project  final  report.  Recovery  of  these  items  provides  a  more  meaningful  evaluation  of  the 
MTADS  and  ORAGS  surveys  because  they  sample  the  inventory  of  targets  that  characterize  the 
true  UXO  threat  on  this  range.  Of  the  189  dug  targets  with  a  documented  record,  91  were  either 
intact  UXO  or  substantial  parts  of  UXO  items.  This  dig  program  resulted  in  slightly  fewer  than 
2.1  digs  per  recovered  UXO.  Even  though  this  was  not  a  comprehensive,  random  sampling  of 
the  primary  dig  lists,  the  false-alarm  rate  was  very  low. 

6.3.3  The  Dewatering  Ponds 

A  total  of  47  seed  targets  were  emplaced  in  the  five  ponds,  including  81 -mm  mortars  and 
105-mm  and  155-mm  projectiles.  The  edges  of  the  ponds,  particularly  of  the  large  pond,  were 
heavily  contaminated  with  large  ferrous  clutter  items.  The  banks  of  the  large  pond  were  about 
2  m  above  the  water  level,  making  it  hard  to  survey  at  low  altitude  near  the  shoreline.  The  ponds 
were  reported  to  be  about  2  m  deep,  but  this  has  not  been  verified.  Table  15,  derived  from  the 
IDA  report,  shows  the  detection  efficiency  for  the  MTADS  survey.  The  MTADS  target  report 
contained  224  targets.  It  was  noted  in  the  NRL  submission  that  about  one-half  of  the  reported 
targets  are  outside  the  shorelines  of  the  ponds  or  are  much  too  large  to  be  155-mm  projectiles. 
These  targets  were  included  in  the  target  list  in  case  APG  wishes  to  investigate  them  sometime  in 
the  future. 

The  coordinates  and  identifications  of  the  seed  targets  are  provided  in  Table  16,  which  also 
provides  information  on  the  targets  that  were  detected  in  the  MTADS  survey. 

The  81  -mm  mortars  were  uniformly  undetectable.  All  of  the  105-mm  and  155-mm  projectiles 
were  detected  in  the  small  ponds;  only  a  fraction  were  detectable  in  the  large  pond.  Of  the 
unreported  targets  in  the  large  pond,  most  were  missed  because  their  signals  were  too  small.  One 
target  (FP-105MM  2)  was  missed  because  its  easting  coordinate  had  been  recorded  incorrectly  in 
the  target  report.  A  few  of  the  targets  were  missed  because  their  signals  were  buried  by  the  very 
large  signal  returns  from  the  edges  of  the  large  pond.  In  addition,  it  is  possible  that  a  few  of  the 
targets  may  have  had  their  coordinates  recorded  incorrectly  or  that  they  were  inadvertently 
moved.  This  is  postulated  because,  in  a  few  cases,  appropriate  signals  were  observed  in 
somewhat  displaced  positions  from  the  reported  coordinates  (e.g.,  P-105MM  12,  P-155MM  1,  P- 
155MM2). 

The  helicopter  altitude  above  the  large  and  small  ponds  was  very  similar.  It  is  likely  that  the 
majority  of  the  targets  were  missed  in  the  large  pond  because  the  water  was  deeper  than  in  the 
smaller  ponds. 
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6.4  TECHNOLOGY  COMPARISON 


Some  aspects  of  the  relative  performances  of  the  ORNL  and  NRL  systems  have  been  treated 
comprehensively  by  an  unbiased  analyst  in  the  IDA  reports.  For  more  detail,  see  NRL’s 
Airborne  UXO  Surveys  Using  the  MTADS  and  the  corresponding  ORNL  report  (currently  in 
draft  form).  These  reports,  in  final  fonn,  will  be  posted  on  the  ESTCP  documents  Web  site. 

Production  cost  information  at  APG  based  on  the  NRL  demonstration  is  not  representative 
because  it  was  effectively  a  local  operation.  By  comparison,  ORNL  mobilized  helicopters  from 
Canada  and  personnel  from  Tennessee. 

Cost  information  derived  from  this  operation  is  not  appropriate  for  evaluation  of  the  Airborne 
MTADS  also  because  of: 

•  The  small  sizes  of  the  survey  areas 

•  The  difficult  access  resulting  from  onerous  security  requirements 

•  The  impracticality  of  the  Active  Recovery  Field  and  offshore  impact  range  sites  from  a 
UXO  survey  point  of  view 

•  The  requirement  of  reanalyzing  data  to  size  limits  far  below  that  dictated  by  ordnance 
requirements 

•  The  extensive  documenting  and  reporting  requirements  that  far  exceed  what  would  be 
required  on  any  realistic  UXO  survey/remediation  operation. 
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7.0  ISLETA  PUEBLO  DEMONSTRATION  DESIGN 


7.1  PERFORMANCE  OBJECTIVES 

The  demonstration  design  for  this  project  included  three  overlapping  surveys. The  first  was  a 
vehicular  magnetometry  survey  of  100  acres  near  the  previously  identified  bull’s  eye,  SI,  on  the 
Pueblo  of  Isleta  near  Albuquerque,  New  Mexico.  The  vehicular  survey  was  to  be  followed  by  an 
Airborne  MTADS  magnetometry  survey  of  1,500  acres  centered  on  the  bull’s  eye  and  including 
the  vehicular  survey  area.  Finally,  the  MTADS  survey  was  to  be  followed  by  an  airborne  survey 
by  ORNL  of  the  same  1,500  acres.  The  selected  survey  areas  are  shown  in  Figure  22.  First- 
order  control  points  were  established  by  a  commercial  surveyor,  GeoMetrics  GPS,  Inc.,  to 
support  this  project  and  other  projects.  The  coordinates  of  the  comer  points  of  the  vehicular  and 
airborne  surveys  are  provided  in  Table  17.  It  was  specified  by  the  ESTCP  Program  Office  that 
the  processing  and  analysis  of  the  vehicular  survey  data  be  handled  entirely  independently  of  the 
airborne  data. 

Based  on  an  earlier  surface  inspection  of  the  area  by  a  senior  UXO  supervisor,  the  primary 
targets  expected  on  this  range  included  M-38  and  BDU-33  practice  bombs.  A  small  amount  of 
heavy-walled  shrapnel  was  observed,  consistent  with  air-dropped  GP  bombs.  In  addition,  the 
ESTCP  Program  Office  specified  that  an  array  of  inert  ordnance  be  emplaced  in  the  vehicular 
survey  area  by  ERDC,  working  with  the  ESTCP  Program  Office. 

The  vehicular  results  were  used  as  a  benchmark  for  the  two  airborne  surveys.  Consequently,  the 
demonstration  test  plan”  specified  that  the  targets  within  the  vehicular  survey  were  to  be 
analyzed  and  fit.  In  practice,  as  discussed  below,  only  ~  69.5  of  the  planned  100  acres  were 
completely  surveyed  by  the  vehicular  MTADS.  The  vehicular  and  airborne  survey  teams 
independently  analyzed  their  data,  prepared  prioritized  target  lists,  and  submitted  the  results  to 
ESTCP  and  the  IDA  at  the  conclusion  of  the  surveys  as  Excel  spreadsheet  target  reports. 

From  these  analyses,  IDA  prepared  an  inclusive  dig  list.  NRL  provided  oversight  to  EOTI,  Inc., 
a  commercial  UXO  remediation  firm,  in  the  reacquisition  of  targets  on  the  dig  list.  After 
reacquisition  and  flagging  of  the  targets  on  the  dig  list,  they  were  excavated.  As  they  were 
uncovered,  targets  were  relocated  using  GPS.  All  target  parameters  were  documented,  and  the 
ferrous  objects  were  photographed. 

The  primary  objective  of  these  demonstrations  was  to  produce  a  quantitative  comparison  of  the 
airborne  systems  to  each  other  and  to  benchmark  their  performance  compared  to  the  vehicular 
MTADS.  A  second  objective  was  to  evaluate  the  airborne  systems’  performance  against 
individual  targets,  including  the  ability  to  distinguish  UXO  from  OE  scrap  and  pre-existing 
clutter. 
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UTM  Northing  (m) 
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Figure  22.  A  Portion  of  a  U.S.  Geological  Survey  (USGS)  Topographic  Map  Showing  the 
Boundaries  of  the  Planned  Surveys.  (The  locations  of  the  two  first-order  points  installed  on  this  site  for 

the  surveys  are  shown  as  1A  and  IB.) 
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Table  17.  Coordinates  for  the  corners  of  the  survey  areas. 


Point 

Latitude 

Longitude 

Northing  (m) 

Easting  (m) 

NAD  83 

Air-NW 

34°  51'  42.726"N 

106°  59'  31.494"W 

3,859,534.87 

317,901.48 

Air-NE 

34°  51'  42.972"N 

106°  58'  08.556"W 

3,859,500.82 

320,007.88 

Air-SE 

34°  50'  09.696"N 

106°  58'  08.724"W 

3,856,627.06 

319,947.15 

Air-SW 

34°  50'  09.576"N 

106°  59'  31.632"W 

3,856,664.97 

317,840.93 

Vehicle-NW 

34°  51'  18.912"N 

106°  59'  05.400"W 

3,858,788.00 

318,549.62 

Vehicle-NE 

34°  51'  19.038"N 

106°  58'  55.650"W 

3,858,786.99 

318,797.32 

Vehicle-SE 

34°  50'  26.694"N 

106°  58'  56.400"W 

3,857,174.63 

318,746.38 

Vehicle-SW 

34°  50'  26.940"N 

106°  59'  06.294"W 

3,857,187.19 

318,495.20 

7.2  SELECTING  THE  TEST  SITE 

The  survey  boundaries  for  this  demonstration  were  chosen  by  the  ESTCP  Project  Office  in 
conjunction  with  the  Environment  Department  of  the  Pueblo  of  Isleta.  The  SI  range  was  chosen 
ahead  of  others  on  the  Pueblo  because  it  is  of  most  concern  to  the  Tribe.  It  has  the  greatest 
probability  of  containing  live  dud  ordnance,  and  it  offered  the  opportunity  to  survey  the  largest 
area  with  the  available  resources. 

7.3  TEST  SITE  CHARACTERISTICS  AND  HISTORY 
7.3.1  Site  Characteristics 

The  Pueblo  of  Isleta  is  located  approximately  10  miles  south  of  Albuquerque  in  north-central 
New  Mexico.  The  Reservation  is  bordered  on  the  north  by  the  Sandia  Military  Reservation, 
which  includes  Kirtland  Air  Force  Base,  the  Manzano  Mountains  on  the  east,  and  the  Rio  Puerco 
and  the  Laguna  Pueblo  Reservation  on  the  west.17 

The  site  consists  of  relatively  flat  terrain;  it  is  primarily  desert  grassland  with  the  elevation 
increasing  from  5,100  ft  on  the  west  to  5,400  ft  above  sea  level  at  a  broken  escarpment  on  the 
east. 


7.3.2  Site  History 

The  area  referred  to  as  Site  B  in  the  Draft  Site  Assessment  Report,17  which  contains  target  SI, 
comprises  an  area  of  approximately  7,000  acres.  This  land  was  leased  from  the  Tribe  in  the 
1950’s  for  use  as  a  bombing  range  for  aircraft  from  Kirtland  AFB.  Documentation  in  Bureau  of 
Indian  Affairs’  files  indicates  that  the  area  was  used  as  a  practice  bombing  range  from  1956  to 
1961,  primarily  for  training  with  fast  aircraft  during  bombing  runs.  In  the  1960s,  Kirtland 
collected  and  piled  surface  ordnance  debris  on  site  for  removal.  Up  to  two  tons  per  acre  of 
practice  bombs  and  ordnance  scrap  were  removed,  but  there  is  no  record  of  intact  explosive 
ordnance  recovery. 
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7.4  PHYSICAL  SETUP  AND  OPERATION 


7.4.1  Predemonstration  Activities 


The  vehicular  MTADS  and  components  of 
the  airborne  system  were  mobilized  to  the 
target  SI  site  using  a  rented  5 3 -ft  trailer, 
Figure  23.  The  MTADS  tow  vehicle,  the 
magnetometer  trailer,  and  vehicular  and 
airborne  components,  spares,  and  supplies 
were  also  transported  in  the  trailer.  The 
helicopter  was  ferried  to  the  site  by 
Helicopter  Transport  Services  from  their 
FBO  hanger  at  the  Martin  State  Airport  in 
Baltimore,  Maryland. 


NRL  rented  all  logistics  supplies,  facilities, 
and  equipment  from  firms  in  Albuquerque. 
One  trailer  was  used  exclusively  for  data 
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Figure  23.  The  MTADS  Base  Camp  for  the  Isleta 
Demonstration  (showing  the  office  and  garage  trailers, 
generator,  diesel  tank,  and  transport  trailer). 

processing  and  analysis,  as  a  communications 
center,  for  battery  storage  and  charging 
stations,  as  an  electronics  repair  station,  and 
for  storing  spares  and  supplies.  A  second  8  ft 
x  48  ft  trailer,  which  could  be  opened  from 
either  end,  was  used  as  a  garage  and  for 
secure  storage  of  the  MTADS  vehicle  and 
sensor  platform.  A  diesel  field  generator 
provided  power  to  the  trailers.  Fuel  storage 
was  provided  for  the  generator,  and  portable 
toilets  were  provided  for  staff.  Figure  23 
shows  the  MTADS  base  camp.  Aviation  fuel 
to  support  the  airborne  survey  was  also 
located  on  site. 


3856000 


7.4.2  Period  of  Operation 


318000  319000  320000 

utm  Easting  The  NRL  portion  of  the  demonstration  was 

accomplished  February  19-27.  The  start  of 
Figure  24.  Planned  Layout  of  the  Isleta  Airborne  ,  .  ,  -  .  , 

Survey.  (The  planned  vehicular  MTADS  survey  the  Survey  waS  delayed  2  dayS  due  to  snow 
bounds  are  shown  in  black.)  on  the  East  Coast  that  closed  area  airports  for 

several  days.  The  vehicular  survey  was 
tenninated  earlier  than  planned  because  of  equipment  failure.  The  survey  log  for  the  airborne 
survey  is  given  in  Table  18  and  for  the  vehicular  survey  in  Table  19.  The  airborne  survey  area 
was  divided  into  12  sorties  of  25  survey  lines  each  (175  m  east  to  west).  These  sorties  and  their 
relation  to  the  vehicular  site  are  shown  schematically  in  Figure  24. 


56 


Table  18.  Survey  Log  and  Production  Information  for  the  Airborne  MTADS  Survey. 


Date 

Activity 

Survey 
File  Name 

Duration 

(min.) 

Wednesday 

MTADS  personnel  arrive  at  site,  unpack  trailer,  and  set  up  office. 

2/19/03 

Transport  airborne  components  to  Belen,  NM,  airport  and 

assemble  sensor  boom. 

Thursday 

Aircraft  arrives  in  Albuquerque.  Mate  survey  hardware  to  aircraft. 

2/20/03 

Friday 

Ferry  aircraft  to  Belen.  High  winds  prevent  survey.  Test  flight 

03053004 

14 

2/21/03 

conducted  late  in  the  day. 

Saturday 

Replace  mag  sensor  #6. 

2/22/03 

Survey  tracks  1-15  of  sortie  7. 

49 

34 

Survey  vehicular  site  (tracks  23-25  of  sortie  7  and  all  of  sortie  8). 

30354003 

56 

30354004 

59 

03054005 

51 

Sunday 

Survey  all  of  sortie  9. 

03355003 

61 

2/23/03 

03355004 

20 

03055005 

60 

Survey  tracks  15-23  of  sortie  7. 

03055006 

51 

Test  flight  for  eastern  edge  of  site.  Track  1  of  sorties  1  and  2. 

03055007 

14 

Survey  all  of  sortie  3. 

03055008 

03055009 

03055010 

Survey  tracks  1-17  of  sortie  4. 

24 

57 

Monday 

Survey  tracks  15-25  of  sortie  4. 

03056001 

47 

2/24/03 

Survey  sortie  5. 

03056002 

45 

03056003 

43 

03056004 

29 

Survey  sortie  6. 

03056005 

61 

03056006 

19 

03056007 

37 

Survey  tracks  1-10  of  sortie  10. 

03056008 

47 

Tuesday 

Survey  tracks  8-25  of  sortie  10. 

60 

2/25/03 

17 

Survey  sortie  1 1 . 

03057003 

39 

03057004 

45 

03057005 

28 

Survey  sortie  12. 

03057006 

03057007 

Survey  sortie  13. 

03057009 

45 

03057010 

9 

03057011 

45 

03057012 

34 

Resurvey  tracks  1 1  and  12  of  sortie  3. 

03057013 

15 

Remove  equipment  from  aircraft. 

Aircraft  departs  site  for  ferry  home. 
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Table  19.  Survey  Log  and  Production  Information  for  the  Vehicular  MTADS  Survey. 


Date 

Activity 

Survey  File 
Name 

Duration 

(min.) 

Monday 

2/24/03 

Static  test. 

03055001 

26 

Site  survey. 

03055002 

55 

Tuesday 

2/25/03 

Site  survey. 

03056001 

31 

03056002 

28 

03056003 

58 

03056004 

60 

03056005 

58 

Calibration  area. 

03056006 

61 

Site  survey. 

03056007 

17 

03056008 

55 

03056009 

58 

Wednesday 

2/26/03 

Site  survey. 

03057001 

58 

03057002 

19 

03057003 

59 

03057004 

50 

03057005 

29 

03057007 

58 

03057008 

15 

03057009 

57 

03057010 

31 

Calibration  area  infill. 

03057011 

2 

03056004  infill. 

03057012 

4 

Thursday  2/27/03 

Site  survey. 

03058001 

60 

03058003 

61 

03058004 

61 

03058005 

63 

03058006 

58 

03058007 

52 

03058008 

30 

Sensor  boom  delaminates,  survey  terminated. 

Friday  2/28/03 

Pack  equipment  for  shipment. 

MTADS  personnel  depart  site. 

7.4.3  Area  Characterized 

The  vehicular  MTADS  survey  covered  28.1  hectares  (~  69.5  acres),  including  a  10  m  buffer 
beyond  the  survey  boundary;  see  Figure  25.  The  vehicular  analysis  spreadsheet  contained  1,364 
targets  including  16  calibration  targets  that  were  planted  near  the  northern  edge  of  the  site. 
Target  analyses  used  the  six-category  probability  classification  scheme.  The  vehicular  picks  are 
given  in  Table  20. 
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Table  20.  Vehicular  MTADS  target  picks  for  the  Isleta  vehicular  survey  area. 


UXO  Classification 

Calibration 

Targets 

1 

2 

3 

4 

5 

6 

Total 

Number  of  picks 

16 

305 

328 

322 

239 

137 

17 

1,364 

The  Airborne  MTADS  surveyed  570  hectares  (1,408  acres).  The  terrain  and  tree  cover  on  the 
two  easternmost  sorties  would  have  required  a  survey  at  greater  than  3  m  above  the  ground. 
Flying  at  this  altitude  would  have  compromised  our  ability  to  detect  the  M-38  and  BDU-33 
ordnance  that  were  the  expected  targets  of  the  survey.  As  the  MTADS  was  flying  the  survey,  the 
ORNL  team  was  still  surveying  on  the  eastern  area  above  the  treetops.  To  maximize  the  useful 
survey  data  for  the  Tribe,  we  deleted  sorties  1  and  2  (see  Figure  24)  and  added  a  new  sortie  on 
the  western  edge  of  the  site.  This 
enabled  us  to  cover  almost  to  the 
western  edge  of  the  Tribal  land 
associated  with  target  S 1 . 


Airborne  UXO  targets  were  picked 
in  two  areas.  The  first  area  was  part 
of  the  100-acre  site  that  was 
surveyed  by  the  vehicular  MTADS 
(see  Figure  25);  the  analyzed  area 
did  not  include  the  densest  area  of 
the  bull’s  eye.  The  targets  were 
picked  and  the  target  list  submitted 
to  ESTCP  before  the  vehicular 
survey  began.  Later,  the  ESTCP 
Program  Office  requested  that  the 
airborne  analyst  pick  more  targets  by 
analyzing  areas  closer  to  the  bull’s 
eye.  In  response  to  this  request,  the 
airborne  analyst,  who  was  not  on  site 
during  the  vehicular  data  collection 
and  had  no  access  to  the  vehicular 
data,  expanded  the  analyzed  portion 
of  the  100-acre  site.  This  resulted  in 
a  target  list  containing  1,260  picks, 
which  are  categorized  in  Table  21. 
The  analyzed  airborne  survey  area  is 
contained  in  the  two  smaller  yellow 
rectangles  north  and  south  of  the 
bull’s  eye,  as  shown  in  Figure  26. 


Figure  25.  Magnetic  Anomaly  Map  from  the  Vehicular 
MTADS  Survey  Superimposed  on  the  USGS  Topo  Map  of  the 

Area. 
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Table  21.  Airborne  MTADS  Target  Picks  Sorted  by  Classification 


UXO  Classification 

Calibration 

Targets 

1 

2 

3 

4 

5 

6 

Total 

Vehicular  area  picks 

12 

502 

336 

282 

42 

69 

17 

1,260 

Primary  area  picks 

93 

85 

70 

48 

52 

40 

388 

318000  318500  319000  319500  320000 


318000  318500  319000  319500  320000 


NAD83  UTM  Zone  13  metres 


Total  Magnetic  Field 

NRL  MTADS  Airborne  Adjunct 
Isleta  UXO  Survey  Feb  2003 


-11- 


Fig  ure  26.  Magnetic  Anomaly  Map  of  the  Isleta  Airborne  Survey.  (The 
vehicular  survey  areas  are  outlined  by  the  smaller  yellow  rectangles.  The 
primary  area  is  outlined  by  the  large  yellow  rectangle.) 
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7.4.4  Areas  Remediated 


Targets  were  remediated  in  the  areas  described  above  as  the  vehicular  area  and  the  Primary  Area. 
The  perimeter  of  the  vehicular  area  is  bounded  in  black  in  Figure  25,  which  also  shows  the  actual 
area  covered  by  the  vehicular  survey.  In  Figure  26,  the  remediation  areas  are  shown  bounded  in 
yellow.  The  larger  Primary  Area  was  surveyed  only  by  the  airborne  systems.  Targets  were 
remediated  in  two  parts  of  the  vehicular  area  contained  within  the  smaller  yellow  rectangles  in 
Figure  26. 

7.4.5  Operating  Parameters  for  the  Technology 

The  Airborne  MTADS  survey  production  data  is  presented  in  Table  18.  Table  22  summarizes 
the  helicopter  use  infonnation.  Staging  the  helicopter  at  the  Belen  airport  and  establishing  a  Jet 
A  fuel  tanker  on  the  survey  site  minimized  local  ferry  times  and  costs.  The  ORNL  survey  team 
was  still  surveying  on  February  22  when  our  production  survey  operations  began.  All  Airborne 
MTADS  survey  operations  were  completed  between  February  22-25.  The  survey  production 
rate  was  1,408  acres/24.1  hours  or  58.4  acres/hour,  based  on  actual  survey  time,  or  49.9 
acres/hour,  including  the  local  ferry  and  test  hours. 


Table  22.  Helicopter  Use  Time,  Based  on  the  Pilot  Log. 


Date 

Mobilize/ 

Demobilize 

On  Survey 

Local 

Ferry/Test 

Total 

Firs 

Firs 

Firs 

Firs 

Feb  18-20 

15.6 

15.6 

Feb  21 

1.3 

1.3 

Feb  22 

4.2 

0.5 

4.7 

Feb  23 

7.0 

1.0 

8 

Feb  24 

5.5 

0.5 

6 

Feb  25 

7.6 

0.6 

8.2 

Feb  26  -  Mar  1 

16.8 

16.8 

Total  log  hours 

32.4 

24.3 

3.9 

60.6 

7.4.6  Survey  Experimental  Design 

A  strict  arm’s-length  relationship  was  maintained  between  the  NRL  vehicular  and  airborne 
surveys.  The  vehicular  data  processor  (who  was  also  the  target  analyst)  was  resident  on  the 
survey  site  during  the  vehicular  survey  operations.  The  airborne  data  processor  (a  different 
person)  was  resident  on  the  survey  site  only  during  the  airborne  survey.  The  airborne  survey 
operations  in  the  vehicular  survey  area  (sorties  7,  8,  and  9)  were  completed  before  the  vehicular 
survey  began,  and  the  data  were  handed  off  to  the  airborne  target  analyst,  who  was  never  present 
on  the  site.  The  airborne  target  analysis  of  the  100-acre  vehicular  area  was  completed  and  the 
target  list  submitted  to  ESTCP  and  IDA  before  the  vehicular  survey  began.  Subsequently,  the 
ESTCP  Program  Office  requested  that  the  survey  analysis  area  be  extended  to  include  areas 
closer  to  the  bull’s  eye.  The  expanded  target  list  was  resubmitted  to  ESTCP  and  IDA;  this  is  the 
data  summarized  in  Table  2 1 . 
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8.0  ISLETA  PUEBLO  PERFORMANCE  ASSESSMENT 


8.1  PERFORMANCE  DATA 


The  100-acre  vehicular  demonstration  area  was  surveyed  by  the  airborne  system  as  part  of  the 
entire  site,  as  shown  in  Figure  26.  All  airborne  survey  lines  followed  the  3,000-m  north-south 
traverses.  Conversely,  the  vehicular  survey  was  conducted  using  250-m  east-west  traverses. 
Figure  27  shows  a  60  m  x  90  m  area  about  1,000  m  north  of  the  bull’s  eye  common  to  both 
surveys.  This  1.3-acre  area  contains  five  of  the  seed  targets — one  81 -mm  mortar  (yellow  circle) 
and  four  105-mm  projectiles  (yellow  pentagons).  In  addition,  an  Mk-76  practice  bomb  (target 
250  in  the  airborne  survey)  is  shown  in  the  yellow  rectangle.  Targets  are  much  denser  nearer  the 
bull’s  eye.  After  the  ESTCP  Program  Office  requested  that  the  airborne  analysis  be  extended  to 
include  areas  closer  to  the  bull’s  eye,  a  total  of  28  ha  (~  69.5  acres)  was  analyzed.  The  expanded 
target  report  contained  1,364  entries  encompassing  the  area  that  included  all  the  emplaced  seed 
targets. 


The  vehicular  MTADS  system  broke  down  during  the  survey,  leaving  30.5  acres  of  the  intended 
area  unsurveyed.  The  area  surveyed  by  the  vehicular  system  included  only  47  of  the  112 
emplaced  seed  targets. 


Figure  27.  Magnetic  Anomaly  Images  from  the  Airborne  Survey  (on  the  left)  and  the  Vehicular 

Survey  (on  the  right). 


8.2  PERFORMANCE  CRITERIA 

The  airborne  demonstration  was  evaluated  from  three  different  perspectives  following  the 
evaluation  criteria  used  in  the  IDA  report: 

•  The  airborne  systems’  performances  were  evaluated  against  the  emplaced  seed  targets  by 
using  the  vehicular  survey  as  a  benchmark.  This  enabled  comparison  of  the  relative 
performances  of  the  two  airborne  systems  and  comparison  of  the  vehicular  and  Airborne 
MTADS  systems  (see  Section  8.3.1). 
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•  Extensive  targets  were  dug  in  the  vehicular  survey  area  that  were  common  to  all  three 
survey  systems.  All  category  1  and  2  targets  from  the  vehicular  survey  area  were  dug,  in 
addition  to  a  few  large  Airborne  MTADS  targets.  There  were  338  recovered  items  in 
these  two  categories.  Relative  system  performances  are  discussed  in  Section  8.3.2. 

•  In  the  analysis  region  (see  Figure  26)  referred  to  as  the  Primary  Area,  161  items  were  dug 
that  were  common  to  the  NRL  and  ORNL  surveys.  These  items  were  chosen  largely 
from  the  category  1  and  2  targets  on  the  NRL  and  ORNL  dig  lists.  The  results  are 
discussed  in  Section  8.3.3. 

8.3  DATA  ASSESSMENT 

8.3.1  Performance  Against  Emplaced  Targets 

Table  23,  adapted  from  the  IDA  report,  shows  the  types  and  numbers  of  seed  targets  that  were 
implanted  and  the  numbers  that  were  detected  by  the  three  surveys.  The  evaluation  assumes  a 
radius  of  1 .5  m  around  the  target  to  qualify  as  a  detection.  This  radial  area  is  referred  to  as  the 
detection  halo.  The  vehicular  MTADS  target  list  had  104  items,  and  the  Airborne  MTADS 
(covering  a  larger  area)  had  165.  Since  the  airborne  system  was  not  designed  to  detect  reliably 
ordnance  smaller  than  2.75-in  warheads,  IDA  expunged  the  60-mm  mortars  from  the  list  before 
constructing  detection  ROC  curves.  Figure  28  shows  ROC  curves,  which  are  an  adaptation  from 
the  IDA  report  based  on  a  1.5-m  detection  halo  and  the  exclusion  of  the  60-mm  mortars  from  the 
seed  target  database.  The  Airborne  MTADS  detected  77%  of  the  larger  2.75-in  and  105-mm 
ordnance. 


Table  23.  Emplaced  Ordnance  Detection  by  Type  for  a  1.5-m  Halo. 


Ordnance 

Total 

Implanted 

Airborne 

MTADS 

Vehicular 

MTADS 

2.75  in 

12 

11 

2  of  2 

60  mm 

20 

4 

6  of  6 

81  mm 

40 

19 

20  of  21 

105  mm 

40 

29 

17  of  18 

Total 

112 

63 

45  of  47 

An  unexpectedly  large  fraction  of  the  105-mm 
targets  was  not  successfully  detected  in  the 
Airborne  MTADS  analysis.  These  were 
missed  for  a  variety  of  reasons.  Two  of  the 
105-mm  seed  projectiles  were  outside  the 
analyzed  survey  area.  The  remaining  nine 
were  missed  because  of  geological 
interferences  or  because  their  signals  were  too 
small  to  measure  due  to  orientation,  burial 
depth,  or  helicopter  altitude.  The  target 
density  as  the  bull’s  eye  was  approached 
became  much  higher,  and  target  signals  began 


false  alarms  (hectare'1) 

Figure  28.  ROC  Curves  for  Emplaced  Ordnance 
Detection  for  a  1.5  m  Halo. 
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to  merge  in  the  Airborne  MTADS  data  set.  This  required  that  the  analysis  display  scale  be 
expanded,  which  resulted  in  loss  of  the  lower-signal  targets. 

The  mean  of  the  target  location  accuracy  of  the 
Airborne  MTADS  was  <  6  cm,  and  the  standard 
deviation  was  ~  30  cm.  Figure  29  shows  a 
scatter  plot  for  the  location  error  for  the  seed 
targets.  These  data,  of  course,  reflect  both  the 
accuracy  of  the  target  emplacement  operation 
and  the  location  accuracy  of  the  survey  and 
analysis  processes.  The  very  small  values  for 
the  locus  of  the  errors  indicate  that  there  is  no 
significant  offset  bias  in  either  of  the  processes. 

The  target  location  accuracy  of  ~  30  cm  is 
consistent  with  the  Airborne  MTADS 
performance  at  other  sites. 

8.3.2  Vehicular  Area  Remediated  Targets 

A  dig  list  was  developed  to  support  recovery  of  targets  from  the  area  surveyed  and  analyzed  by 
each  of  the  three  survey  systems.  The  list  is  comprised  primarily  of  targets  from  categories  1  and 
2  of  the  vehicular  MTADS  dig  list.  About  a  dozen  targets  categorized  as  large  bombs  were 
added  from  the  Airborne  MTADS  dig  list. 

The  IDA  report  considered  the  results  of  272  digs  from  this  list.  Only  a  fraction  of  the  targets 
labeled  as  very  large  and  deep  on  the  Airborne  MTADS  list  were  dug  because  of  the  time 
required  to  exploit  the  large,  deep  targets.  Of  the  six  targets  dug  that  did  not  appear  on  the 
vehicular  MTADS  dig  list,  three  were  500-lb  or  1,000-lb  bombs  and  three  were  categorized  by 
the  dig  team  as  “nuclear  simulator  shapes.”  In  the  IDA  report,  the  dug  targets  were  divided  into 
the  five  categories  shown  in  Table  24.  Figure  30  shows  the  IDA  ROC  curves  for  the  system 
performances  based  on  a  1.5-m  halo.  The  detections  and  background  alarm  rates  are  based  on 
1,136  declarations  by  the  Airborne  MTADS  team,  and  1,237  by  the  vehicular  MTADS  team.  In 
this  figure,  both  intact  ordnance  and  ordnance-related  scrap  are  categorized  as  ordnance 
detections. 

Table  24.  Results  of  the  Ordnance  Remediation  Operation  in  Vehicular  Survey  Area. 
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Figure  29.  Airborne  MTADS  Location  Error 
Scatter  Plot  for  the  Seed  Targets  for  a  1.5  m  Halo. 


Vehicular 

MTADS 

Classification 

Intact 

Ordnance 

Ordnance 
Related  Scrap 

Nonordnance 
Related  Clutter 

Geology 
“Hot  Dirt” 

Empty 

Hole 

1 

53 

160 

7 

1 

0 

2 

6 

27 

4 

2 

0 

4 

0 

1 

0 

0 

0 

Total 

59 

188 

11 

3 

0 

There  is  an  excellent  correlation  between  the  MTADS  airborne  and  vehicular  versions  of  the  dig 
lists  that  compose  this  list  of  remediated  targets,  in  part  because  the  dig  list  was  prepared  from 
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category  1  and  2  targets  from  the  vehicular  list. 

If  both  the  “intact  ordnance”  and  “ordnance- 
related  scrap”  categories  are  included  as 
ordnance,  they  constitute  94.6%  of  the  list;  23% 
of  the  dug  targets  on  the  list  were  intact  UXO. 

The  remediated  targets  were  located  with  GPS 
when  they  were  uncovered  so  their  locations 
could  be  precisely  determined.  They  were 
photographed  and  either  removed  or  blown  in 
place  if  it  was  decided  that  they  should  not  be 
moved.  Detennining  the  locations  of  the  targets 
as  they  were  recovered  enabled  an  evaluation  of 
the  location  accuracy  of  the  MTADS  surveys 
and  analyses.  Figure  31  shows  the  distribution. 

These  plots  bin  the  detections  into  a  histogram 
and  show  the  distribution  and  90%  and  95% 
recovery  points.  The  location  accuracies  are 
somewhat  lower  than  those  attained  for  the  seed 

targets.  This  is  understandable  because  most  of  the  targets  on  this  dig  list  were  large;  many  were 
broken  up  or  were  located  in  the  midst  of  clutter  from  bomb  fragments. 


Figure  30.  ROC  Curves  for  Targets  Remediated 
in  the  Vehicular  Area  for  a  1.5  m  Halo. 
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Figure  31.  Histogram  Plots  Showing  Location  Accuracies  of  the  Vehicular  and  Airborne  MTADS 
Remediated  Targets  in  the  Vehicular  Area  (using  a  1.5  m  halo). 


8.3.3  Targets  Remediated  in  the  Primary  Area 

A  dig  list  was  prepared  from  the  category  1  and  2  targets  from  the  MTADS  and  ORNL  target 
reports  for  the  Primary  Area.  NRL  reported  366  targets  in  this  area.  In  the  Primary  Area,  a  total 
of  338  targets,  which  had  been  reacquired  with  GPS,  were  dug  and  photographed.  Figure  32 
shows  the  IDA  ROC  curves  for  these  digs,  assuming  a  1.5-m  halo.  Scoring  was  done  twice, 
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once  with  only  intact  ordnance  contributing 
to  detection  and  once  with  both  intact 
ordnance  and  ordnance  scrap  contributing  to 
a  positive  declaration.  These  are  identified  as 
case  1  and  case  2  in  Figure  32.  Table  25 
shows  the  target  location  error  statistics  for 
these  recovered  targets.  The  location 
accuracies  are  similar  to  those  for  the  targets 
dug  in  the  vehicle  survey  area. 

8.3.4  Reinvestigation  of  “No  Finds”  in 
the  Primary  Area 


1.0 


Of  the  318  targets  that  were  remediated  in  the 
Primary  Area,  196  appear  on  the  NRL  target 
report.  Of  these  196,  61  were  negative  finds, 
i.e.,  declared  as  either  “empty  holes”  or 
geology/hot  dirt.  Because  the  dig  results 
reported  an  unusually  high  proportion  of  empty  holes,  and  because  very  few  digs  in  the  vehicular 
area  were  declared  as  empty  holes  it  was  decided  to  selectively  reinvestigate  a  portion  of  the 
empty  holes.  Both  the  NRL  and  ORNL  airborne  analysts  were  asked  to  suggest  a  list  of  20 
empty  holes  that  merited  reinvestigation. 


0  1  2  3  4  5 

background  alarm  rate  (#/hectare) 

Figure  32.  ROC  curves  for  the  targets  remediated 
in  the  Primary  Area  for  a  1.5  m  halo. 


Table  25.  Location  Error  Statistics  for  the  Primary  Area  for  a  1.5-m  Halo. 


Mean  Error 

(cm) 

Std.  Dev.  (cm) 

System 

North 

East 

North 

East 

Airborne  MTADS 

5 

9 

31 

30 

A  new  investigation  list  was  prepared  from  the  NRL  and  ORNL  suggestions,  and  these  targets 
were  reinvestigated  using  the  NRL  man-portable  magnetometer  system.  Reinvestigated  targets 
with  a  positive  signal  response  were  dug  again;  22  NRL  airborne  targets  were  redug.  As  a  result 
of  these  redigs,  one  BDU-33  was  recovered,  one  hole  was  found  to  contain  metallic  OE  scrap, 
and  six  holes  were  found  to  hold  metal  scrap  that  was  not  OE.  The  remaining  holes  were 
declared  to  contain  magnetic  soil/hot  rocks  or  to  be  empty.  Many  of  the  geology  returns 
produced  signatures  in  the  original  analysis  that  very  closely  approximated  UXO  targets. 

8.4  TECHNOLOGY  COMPARISON 

Certain  aspects  of  the  relative  performances  of  the  ORNL  and  NRL  systems  have  been  treated 
comprehensively  by  unbiased  analysts  in  the  IDA  reports.  More  information  is  available  in  the 
Isleta  demonstration  reports  prepared  by  each  team.  See  NRL’s  Airborne  UXO  Surveys  Using 
the  MTADS  and  the  corresponding  ORNL  demonstration  report,  which  is  still  in  draft  form. 
These  reports  will  be  posted  on  the  ESTCP  documents  Web  site. 
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Table  26  compares  performances  in  detecting  emplaced  targets  for  the  vehicular  and  airborne 
MTADS  for  a  1.5  meter  halo;  corresponding  IDA  ROC  curves  are  shown  in  Figure  28.  Table  26 
provides  the  mean  location  errors  and  standard  deviations  for  each  system. 


Table  26.  Location  Error  Statistics  for  the  Vehicular  Area  Using  a  1.5-m  Halo. 


Mean  Error 
(cm) 

Std.  Dev  (cm) 

System 

North 

East 

North 

East 

Airborne  MTADS 

2 

6 

39 

35 

Vehicular  MTADS 

1 

6 

26 

25 

Figure  32  plots  the  ROC  curves  for  the  airborne  system  for  targets  remediated  in  the  Primary 
Area.  Location  error  statistics  for  the  same  area  are  provided  in  Table  25. 

Direct  comparison  of  the  detection  efficiencies  of  the  vehicular  and  Airborne  MTADS  systems 
for  the  seed  targets  is  not  particularly  informative  as  the  vehicular  system  broke  down  after 
covering  only  47  of  the  112  seed  targets  and  the  vehicular  system  was  designed  to  detect  the  60- 
and  81 -mm  mortars  (~  54%  of  all  the  seed  targets)  while  the  Airborne  MTADS  was  not.  The 
Airborne  MTADS  correctly  reported  approximately  50%  of  the  81 -mm  mortars  and  77%  of  the 
larger  seed  targets.  The  average  target  location  accuracy  for  the  airborne  system  was  ~  30  cm  for 
the  detected  seed  targets,  which  is  consistent  with  the  prior  demonstrations. 

In  the  three-system,  common-area  dig  list  prepared  primarily  from  the  vehicular  category  1  and  2 
targets,  the  detection  efficiencies  of  the  vehicular  and  airborne  MTADS  systems  were 
indistinguishable,  as  shown  in  the  ROC  curves  in  Figure  30.  The  relative  location  accuracies 
were  ~  26  cm  and  ~  36  cm  for  the  vehicular  and  airborne  systems  for  all  258  of  the  digs  that 
contained  metallic  objects. 

The  relative  production  costs  for  the  vehicular  and  Airborne  MTADS  surveys  are  treated  in 
detail  in  the  final  report,  Airborne  UXO  Surveys  Using  the  MTADS.  In  that  report,  we 
developed  a  cost  estimate  for  a  1500-acre  survey,  typical  of  the  Isleta  survey  described  in 
Sections  7  and  8  of  the  same  report.  The  costs  were  developed  to  compare  an  airborne  survey 
with  a  similar  vehicular  survey.  It  is  assumed  that  mobilization  is  from  the  East  Coast  to  the 
New  Mexico  site.  The  preparation  and  startup  costs,  which  include  site  visits,  establishing 
control  points,  development  of  a  Test  Plan  and  a  Health  and  Safety  Work  Plan,  are  the  same  for 
the  two  approaches.  Mobilization  and  demobilization,  on-site  logistics,  and  survey  support  costs 
are  appropriate  for  each  survey.  Target  analysis  is  based  on  an  assumed  10,000  targets,  with  the 
only  survey  reporting  being  preparation  of  a  target  report  suitable  for  target  reacquisition  and 
support  of  digging  operations.  The  airborne  survey  is  assumed  to  take  4  survey  days,  with  two 
days  of  setup,  teardown,  calibration,  and  training.  The  vehicular  survey  is  presumed  to  require 
75  survey  days  (for  a  single  survey  system).  Operations  are  assumed  to  occupy  a  5-day  week 
with  maintenance  and  repair  taking  place  on  weekends.  Actual  target  analysis  is  assumed  to  take 
place  off  site.  The  costs,  presented  in  Table  27  and  summarized  by  survey  and  support  activities, 
are  organized  somewhat  differently  from  the  data  in  Tables  34  and  35  of  the  final  report.  Fuel, 
labor,  charter  costs,  and  on-site  logistics  are  considered  as  survey  costs  in  Table  27.  On-site 
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setup,  tear  down,  calibration,  and  training  costs  are  considered  as  mobilization/demobilization 
costs,  as  are  equipment  shipping  costs  and  rentals  to  support  site  logistics.  Data  analysis  and 
preparation  of  dig  sheets  and  reports  are  considered  as  survey  costs.  This  breakdown  is  more 
consistent  with  the  way  that  UXO  geophysics  surveys  are  bid  and  costed. 

The  last  two  rows  in  Table  27  are  the  total  survey  costs  or  costs/acre,  the  latter  for  adding  survey 
acres  to  the  hypothetical  survey.  These  costs  are  unrealistic  for  projecting  commercial  survey 
costs  as  they  do  not  include  allowances  for  equipment  capitalization  or  amortization,  provision 
for  fee,  or  allowances  that  are  always  required  to  support  QA  operations  and  required  QC  follow¬ 
ups.  Depending  on  the  circumstances,  these  cost  factors  might  double  the  actual  costs  to  conduct 
the  survey.  The  costs  in  Table  27  do  not  include  provisions  for  weather  delays,  labor  stoppages, 
change  orders,  or  stop-work  orders.  These  provisions  are  typically  negotiated  as  separate  clauses 
in  UXO  survey  bids. 

We  have  conducted  several  airborne  surveys  similar  to  that  addressed  in  Table  27.  The  costs,  as 
projected,  are  realistic  with  the  caveats  cited  in  the  previous  paragraph.  We  have  never 
conducted  a  vehicular  survey  nearly  this  extensive.  It  is  unlikely  that  a  15-week  vehicular  survey 
could  be  conducted  without  significant  down  time  for  unscheduled  maintenance,  emergency 
repairs,  or  other  unforeseen  contingencies.  Indeed,  during  this  demonstration,  the  vehicular 
system  broke  down  before  completing  70%  of  the  100-acre  survey,  causing  the  vehicular  survey 
to  be  terminated.  The  surface  topography  of  this  site  was  more  rugged  than  that  in  the  previous 
BBR  or  APG  demonstrations;  however,  it  was  not  atypical  of  many  of  the  U.S.  western  desert 
ranges. 
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Table  27.  Comparison  of  Survey  Costs  (in  $K,  except  where  indicated)  for  1,500-Acre 
Vehicular  and  Airborne  MTADS  Surveys. 


Preparation  and  startup 
costs 

ESfillH 

iMIWIiWWIM 

Site  visit/inspection 

6 

6 

Test  plan,  maps,  photos 

15 

15 

Establish  control  points 

8 

8 

Total  startup  costs 

29 

29 

Mobilization/demobilization 

costs 

Equipment  to  and  from  site 

4 

21 

Helo  charter,  pilot,  fuel 

26.9 

Helo  charter  for  setup/teardown 

3 

Equipment  repair/restock 

2 

10 

Total  mobilization/demobilization  costs 

35.9 

31 

Survey  costs 

On-site  logistics 

Equipment  rental 

2 

Electrical 

2 

10 

Fuel  (diesel  and  gasoline) 

1 

1 

Materials 

2 

10 

Office,  storage,  trailers,  toilets 

6 

Survey  operations 

Charter 

20.2 

Rental  vehicles 

2 

30 

Fuel 

3 

3 

Labor/per  diem 

21.7 

253.3 

Flazardous  waste  operation 
(HAZWOPR)  labor 

45 

Airfare 

2 

10.7 

Data  analysis,  survey 
products 

20 

20 

Total  survey  costs 

75.9 

389 

Nominal  survey  costs  ($/Acre) 

50.60 

259.33 

Grand  total 

140.8 

449 
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9.0  COST  ASSESSMENT 


9.1  COST  REPORTING 

In  Tables  10  and  11  in  Section  4  and  Table  27  in  Section  8,  we  presented  cost  analyses  for 
hypothetical  1,500-acre  UXO  surveys  with  conditions  similar  to  the  demonstrations  conducted  at 
the  BBR  Impact  Area  and  the  Isleta  SI  target  range.  The  survey  costs  were  comparatively 
developed  assuming  Airborne  MTADS  and  vehicular  MTADS  surveys.  It  was  assumed  that  the 
UXO  survey  products  would  eventually  support  UXO  clearance  operations.  However,  no 
provision  was  made  for  costs  of  resurveys  that  would  be  associated  with  QA  operations  or  QC 
follow-up.  The  former  costs  are  typically  billed  by  the  contractor  as  part  of  the  UXO  survey,  and 
the  QC  follow-up  invariably  requires  additional  geophysical  studies.  These  costs  are  highly 
variable,  depending  on  the  particular  site,  its  level  of  existing  contamination,  and  anticipated 
UXO  clearance.  Because  of  these  uncertainties  and  because  it  is  likely  that  the  commercial 
airborne  UXO  surveys  will  be  used  most  extensively  for  footprint  reduction,  we  have  not 
attempted  to  tackle  these  issues. 

We  have  also  not  treated  the  capital  equipment  as  part  of  the  cost  equation.  Equipment  costs  will 
have  to  be  treated  very  differently  for  equipment  supporting  airborne  and  vehicular  surveys,  even 
though  there  is  much  commonality  between  the  systems.  For  illustration,  assume  that  an 
application  involves  a  10,000  acre  UXO  survey.  At  100  acres  per  week  for  a  single  vehicular 
system,  this  would  require  about  two  years  of  survey  work.  It  is  very  likely  that  the  entire 
vehicular  field  system  would  have  to  be  capitalized  for  this  one  application,  and  moreover  it 
would  require  substantial  additional  (but  unpredictable)  maintenance  and  repair  costs.  The  same 
10,000  acre  survey  with  the  Airborne  MTADS  would  require  5  or  6  weeks  at  400  acres  per  day. 
Assuming  the  pilot  avoids  catastrophic  interaction  with  the  terrain,  the  wear  and  tear  on  the 
airborne  survey  equipment  should  be  minimal.  A  single  airborne  system  operating  40  weeks  per 
year  should  be  able  to  survey  80,000  acres.  If  this  level  of  business  could  be  generated,  the 
amortization  costs  of  the  airborne  equipment  could  be  well  under  $10/acre,  even  if  the  system 
was  amortized  over  a  1-year  period. 

Table  28  shows  the  projected  capital  equipment  costs  associated  with  producing  a  new 
production  system  of  each  type.  These  costs  are  based  on  our  experience  in  the  development  of 
the  vehicular  and  Airborne  MTADS  platforms.  The  initial  cost  outlays  to  acquire  the  systems 
are  very  similar.  We  assume  that  the  helicopter  supporting  the  airborne  platform  is  chartered, 
which  moves  that  expense  to  a  direct  survey  cost.  The  vehicular  platform  tow  vehicle  is 
assumed  to  be  a  John  Deere  Gator  or  equivalent  all-terrain  vehicle  (A TV).  This  cost  is 
substantially  less  than  the  cost  of  the  MTADS  or  Surface  Towed  Ordnance  Locator  System 
(STOLS)  tow  vehicles.  Given  our  experience  with  these  tow  vehicles,  we  feel  that  the  costs 
associated  with  the  very  low  ferrous  content  vehicles  can  be  avoided  because  platform  signatures 
can  be  largely  dealt  with  in  data  processing.  The  tow  vehicles  will,  however,  have  to  be  rugged 
enough  for  extended  use,  capable  of  sheltering  the  computers  and  electronics  from  the  weather, 
and  able  to  carry  batteries  sufficient  to  support  all-day  operation  of  the  survey  equipment. 
Transportation  costs  for  the  entire  vehicular  survey  system  are  likely  to  be  substantially  more 
than  shipment  costs  for  the  airborne  components  but  less  than  the  ferry  costs  for  the  charter 
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helicopter.  In  each  case,  these  costs  are  associated  with  mobilization  and  costed  separately  from 
per-acre  survey  costs. 


Table  28.  Capital  Equipment  Cost  Comparisons  in  $K  for  Airborne  and  Vehicular  UXO 

Survey  System. 


Component 

Airborne  System 

Vehicular  System 

Unit 

Cost 

Platform 

Spares 

Unit 

Cost 

Platform 

Spares 

Sensors/mag 

25 

175 

50 

25 

175 

50 

Sensors/other 

30 

30 

10 

10 

10 

5 

Sensor  platform 

40 

40 

40 

80 

80 

10 

Support  vehicle* 

10 

10 

5 

Vehicle/interface 

mods 

10 

10 

5 

15 

15 

10 

DAQ  hardware 

40 

40 

20 

30 

30 

20 

Sensor-DAQ 

interface 

10 

10 

10 

20 

20 

20 

Navigation 

60 

60 

5 

60 

60 

5 

DAS  hardware 

30 

30 

10 

30 

30 

10 

Software  licenses 

6 

6 

0 

6 

6 

0 

Tools  and 
specialized  field 
support  equipment 

10 

5 

10 

5 

Shipping  containers 

30 

10 

30 

10 

Capital  equipment 
and  spares  costs 

606 

626 

*  Airborne  support  vehicle  is  a  chartered  helicopter. 

Vehicular  platform  is  towed  by  a  customized  John  Deere  Gator  NP  4X4  Trail  ATV. 


9.2  COST  ANALYSIS 

The  cost  drivers  are  very  different  for  Airborne  MTADS  and  vehicular  MTADS  UXO  surveys. 
Vehicular  towed  array  survey  costs  are  dominated  by  labor  costs.  In  the  hypothetical  1,500-acre 
vehicular  survey  described  in  Table  27,  more  than  70%  of  the  actual  survey  costs  are  labor  and 
per  diem  costs.  An  additional  15%  of  the  ancillary  survey  costs  (airfare,  rental  vehicles,  fuel, 
toilets,  etc.)  are  indirectly  associated  with  personnel  on  site.  The  fraction  of  the  survey  costs 
associated  with  labor  would  be  somewhat  lower  for  much  smaller  surveys,  which  could  be 
completed  with  fewer  field  support  staff  and  which  would  not  require  crew  rotations. 

The  comparative  labor  costs  to  support  the  same  survey  with  the  airborne  equipment  are  slightly 
over  30%  of  the  vehicular  equipment  costs.  Data  analysis  and  creation  of  survey  products  were 
not  included  in  these  comparisons  as  their  costs  are  identical  for  both  surveys.  The  primary 
airborne  survey  cost  drivers  are  associated  with  the  charter  and  support  costs  for  the  helicopter 
system,  which  are  about  30%  of  the  actual  survey  costs.  The  fraction  of  the  airborne  survey 
costs  associated  with  the  helicopter  will  remain  relatively  constant  for  surveys  larger  than  ~ 
1,000  acres. 
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9.3  COST  COMPARISON 


In  this  study,  we  have  concentrated  primarily  on  comparisons  between  the  vehicular  and 
Airborne  MTADS  and,  by  extension,  on  hypothetical  commercial  systems  derived  from  these 
systems’  designs.  Table  27  points  out  that  preparation  and  startup  costs,  mobilization  and 
demobilization  costs,  and  data  analysis  and  creation  of  survey  product  costs  are  very  similar  for 
the  airborne  and  vehicular  systems,  even  though  there  are  differences  in  individual  component 
costs.  The  demonstrations  at  the  Impact  Area  of  the  BBR  and  at  target  SI  of  Isleta  have 
projected  that  vehicular  survey  costs  on  a  per-acre  basis  are  about  400%-600%  higher  than  costs 
on  the  same  sites  using  the  airborne  system.  These  estimates  are  also  supported  by  other 
airborne  surveys  conducted  with  the  MTADS  and  by  many  additional  surveys  conducted  with 
the  vehicular  MTADS.  These  cost  comparisons  do  not  include  equipment  capitalization  and 
amortization  costs. 

Table  28  shows  that  capital  equipment  costs  would  be  similar  for  creation  of  new  airborne  and 
vehicular  array  systems  based  on  the  MTADS  designs.  Realistically,  the  vehicular  capital 
equipment  costs  should  be  amortized  over  5,000  acres  of  surveys  (50  weeks  at  100  acres/week). 
Based  on  the  costs  in  Table  28,  this  would  add  approximately  $  125/acre  to  the  vehicular  survey 
costs.  It  seems  reasonable  to  assume  that  the  airborne  system,  being  subject  to  less  wear  and 
tear,  could  be  amortized  over  2  years  of  work.  If  enough  work  could  be  captured  to  support  the 
equipment  at  200  days  per  year,  equipment  costs  could  be  amortized  over  160,000  acres,  which 
would  add  about  $4. 25/acre  to  the  cost  of  the  surveys. 

The  key  driver  in  overall  UXO  remediation  costs  is  the  cost  of  excavation  of  individual  items 
that  pose  no  UXO  risk  to  health  or  safety.  These  costs,  depending  on  the  prior  use  of  the  range 
and  its  potential  future  uses,  may  be  60%-90%  of  the  total  site  cleanup  costs,  with  all  the 
geophysics  operations  being  only  10%-20%  of  the  costs.  Therefore,  the  ability  of  the 
geophysics  survey  and  analysis  to  exclude  harmless  UXO  fragments  and  other  ferrous  scrap 
from  the  dig  list  may  outweigh  the  cost  advantages  of  inexpensive  surveys.  Table  1 1  in  Section 
4.4.3  of  this  report  considered  this  issue  for  the  BBR  demonstration.  At  this  range,  we  concluded 
that  about  50%  more  targets  would  have  to  be  dug  behind  an  airborne  survey  than  behind  a 
vehicular  MTADS  survey.  At  this  relatively  sparsely  populated  range  with  relatively  large 
UXO,  we  still  predicted  a  60%-70%  cost  advantage  for  an  airborne  survey  and  remediation.  The 
results  at  the  APG  survey  were  not  extensive  enough  to  make  these  extrapolations. 

The  results  at  the  Isleta  S 1  survey  indicated  that  remediation  based  on  an  airborne  survey  would 
leave  some  undetected  UXO  in  the  field  that  would  be  detected  by  a  vehicular  towed  array. 
However,  in  the  three-system  common  survey  area,  based  on  the  number  of  targets  picked  and 
the  ability  to  discriminate  between  UXO  and  scrap,  cleanups  following  either  the  NRL  airborne 
or  vehicular  system  would  require  that  similar  numbers  of  targets  be  dug  from  either  analysis.  In 
this  scenario,  a  site  cleanup  based  on  an  airborne  survey  would  be  several  times  less  expensive. 

The  technology  deployment  comparisons  that  we  made  looked  at  tradeoffs  between  the  use  of 
the  vehicular  and  Airborne  MTADS  (or  their  commercial  equivalents).  In  reality,  there  are  very 
limited  circumstances  in  which  these  technologies  should  be  competitively  pitched  against  each 
other.  The  vehicular  system  should  be  considered  when  (1)  the  survey  areas  are  relatively  small 
(a  few  dozen  to  a  few  hundred  acres)  or  (2)  the  site  is  known  to  be  densely  contaminated  with  a 
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wide  range  of  ordnance  sizes  that  must  be  comprehensively  cleared.  The  airborne  survey  system 
should  be  considered  primarily  when  (1)  the  survey  areas  are  relatively  large  (several  hundred  to 
thousands  of  acres),  (2)  large  areas  need  to  be  evaluated  for  footprint  reduction,  or  (3)  site 
conditions  will  not  support  foot  or  vehicular  traffic.  We  anticipate  that  in  many  large  UXO 
survey  areas,  the  most  economical  deployment  of  equipment  will  involve  use  of  both  airborne 
and  vehicular  systems.  The  airborne  system  can  be  used  for  wide  area  coverage  to  identify  areas 
that  must  be  very  carefully  analyzed  for  comprehensive  clearance.  These  areas  could  then  be 
addressed  by  some  combination  of  vehicular  and/or  man-portable  equipment.  In  areas  involving 
very  shallow  water,  swamps,  marshes,  or  wetlands,  the  only  practical  survey  alternative  may  be 
the  airborne  system.  UXO  clearances  in  these  areas  will  be  very  expensive. 
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10.0  IMPLEMENTATION  ISSUES 


10.1  COST  OBSERVATIONS 

The  largest  single  factor  affecting  the  Airborne  MTADS  survey  costs  and  production  rates  is  the 
cost  of  operating  the  survey  helicopter  on  site.  During  recent  surveys,  charter  costs  have  been 
approximately  $700  per  hour  with  a  guaranteed  4-hour  daily  minimum  over  the  duration  of  the 
charter.  Mobilization  of  the  aircraft  to  and  from  the  site,  originating  from  its  home  base,  is 
charged  at  the  hourly  charter  rate.  To  maximize  production  and  minimize  cost,  surveys  should 
be  arranged  with  long  survey  lines  to  minimize  the  time  spent  in  turns.  Frequent  examination  of 
data  quality  minimizes  time  spent  taking  unusable  data.  Minimizing  time  lost  in  refueling 
aircraft  by  having  fuel  available  on  site  and  basing  aircraft  strategically  to  minimize  daily  ferry 
trips  to  and  from  the  survey  site  can  represent  large  increases  in  productivity  and  decreases  in 
production  cost. 

The  take-home  message  from  our  demonstrations  is  that  it  is  unlikely  to  be  economical  to 
undertake  Airborne  MTADS  surveys  of  less  than  a  few  hundred  acres.  Mitigating  circumstances 
occur  when  UXO  surveys  must  be  done  over  water,  in  marshy  wetlands,  or  in  other  areas  where 
one  can  neither  walk  nor  drive.  In  these  situations,  performance  issues  may  override  cost  issues. 

Other  steps  to  maximize  productivity  for  the  Airborne  MTADS  survey  of  the  target  ranges  were 
taken  at  the  BBR,  APG,  and  Isleta  demonstrations: 

•  At  APG,  permission  was  obtained  from  Bell  Helicopter  to  allow  the  helicopter  to  refuel 
with  JP-8  (the  military  equivalent  to  Jet  A).19  Jet  A  was  not  available  at  the  Airfield. 
Refueling  with  JP-8  therefore  required  no  ferry  time.  Refueling  took  place  either 
between  survey  sites  or  when  downloading  survey  data  for  inspection. 

•  At  APG,  the  helicopter  was  chartered  from  Helicopter  Transport  Services  from  their 
Fixed  Base  Operator  (FBO)  hangar  at  Martin  State  Airport  (approximately  20  minutes’ 
flying  time  from  APG).16  The  platform  and  electronics  were  assembled  and  mounted  on 
the  helicopter  at  Martin  State  Airport.  Spares  were  stationed  on  site  to  provide  quick 
recovery,  if  necessary. 

•  At  all  demonstration  sites,  one -hour  missions  were  flown  and  the  resulting  data  provided 
to  analysts  on  the  ground  for  inspection. 

•  At  all  three  demonstration  sites,  survey  missions  were  set  up  in  advance  on  the  DAQ 
computer.  This  enabled  us  to  switch  between  survey  sites,  as  necessitated  by  weather  or 
logistics  (e.g.,  sharing  survey  ranges  with  the  other  demonstrators)  by  simply  starting 
new  survey  files. 

•  At  the  Isleta  demonstration,  a  long  ferry  was  required  originally  to  bring  the  helicopter  to 
the  area.  Rather  than  basing  the  helicopter  at  the  Albuquerque  airport,  we  based  it  at  a 
small  municipal  airport  nearer  the  target  range  to  decrease  daily  ferry  time  to  and  from 
the  site.17  A  fuel  tanker  truck  was  chartered  and  placed  on  the  impact  range  for  refueling. 


75 


•  All  surveys  were  planned  to  start  at  sunup  (or  when  weather  allowed  access)  and  end  at 
sundown  each  day,  with  brief  pilot  rest  breaks  each  hour  and  a  45-minute  break  for  lunch. 

10.2  PERFORMANCE  OBSERVATIONS 

Unlike  the  vehicular  magnetometer  system,  the  airborne  system  is  not  capable  of  detecting  the 
smallest  classes  of  buried  UXO  at  depth.  While  the  magnetic  signals  are  spatially  spread  and 
diminished  in  intensity  with  the  sensors  farther  above  the  ground,  our  modeling  results  indicated 
that,  at  an  altitude  of  2  m  above  the  ground,  the  system  should  be  capable  of  detecting  BDU-33s 
or  Mk  82s  in  all  geologies  and  ordnance  targets  equivalent  to  or  larger  than  2. 75 -in  warheads  in 
geologically  quiet  areas.  This  has  been  borne  out  by  the  demonstrations  described  in  this  report. 
At  the  geologically  quiet  and  topologically  flat  prove-out  site  at  the  Airfield,  we  were  able  to 
detect  efficiently  both  60-mm  and  81 -mm  mortars.  ’  "  At  the  much  more  highly  cluttered  and 
geologically  active  Isleta  range,  in  areas  with  rough  ground  surface  or  significant  vegetation,  we 
failed  to  detect  several  105-mm  projectiles.  “ 

The  extent  to  which  spreading  target  signatures  interfere  with  each  other  and  are  obscured  by 
geological  features  was  carefully  evaluated  in  the  first  airborne  demonstration  at  the  BBR.15  In 
that  study,  with  relative  large  UXO  targets  (105-mm  to  8-in  projectiles)  relatively  sparsely 
distributed  on  the  site,  detection  efficiency  for  individual  UXO  was  equivalent  for  the  airborne 
and  vehicular  towed  arrays.  Because  of  the  lower  data  density  and  the  more  widely  spread 
anomaly  signatures,  it  proved  more  difficult  to  discriminate  between  UXO  and  clutter  signatures 
from  the  airborne  data  than  from  the  vehicular  data.  At  some  APG  sites,16  significantly  more 
targets  would  have  to  be  dug  behind  an  airborne  survey  than  behind  a  corresponding  vehicular 
survey.  At  the  Isleta  site,  if  we  were  cleaning  UXO  from  areas  near  the  bull’s  eye,  more  targets 
would  have  to  be  dug  following  an  airborne  survey  than  would  be  necessary  from  a  vehicular 
survey.  This  results  from  the  much  higher  target  densities  and  the  more  complex  mix  of  UXO 
threats  on  some  of  these  ranges  that  result  in  merging  and  overlapping  of  adjacent  target 
signatures.  The  cost  tradeoffs  between  digging  more  targets  and  reduced  survey  production  costs 
are  (and  always  will  be)  site  specific,  depending  on  the  types  of  UXO  challenges,  the  relative 
density  of  targets,  geological  and  topological  conditions,  and  the  size  of  the  survey  site.  On  all 
sites  in  areas  of  high  OE  concentration,  such  as  around  bull’s  eyes,  or  areas  with  both  large  and 
very  small  ordnance  threats,  multiple  cycles  of  survey  and  dig  will  be  required  to  accomplish  an 
effective  clearance. 

On  large,  open  sites,  such  as  the  Impact  Area  of  the  BBR  or  the  S 1  target  at  Isleta,  the  Airborne 
MTADS  routinely  completed  400  acres  per  day  using  a  two-man  field  crew.  On  the  very  small 
cluttered  and  vegetated  sites  at  APG  with  careful  survey  planning,  we  were  still  able  to  achieve 
production  rates  of  30-40  acres  per  survey  hour.  The  Airborne  MTADS  is  a  very  productive 
UXO  survey  platfonn. 

10.3  SCALE-UP 

The  end  user  of  the  Airborne  MTADS  technology  is  most  likely  to  be  one  or  more  of  the  large 
architectural  and  engineering  (A&E)  firms  that  do  substantial  amounts  of  UXO  geophysics  work. 
With  some  consulting  cooperation  with  the  original  developers,  the  Airborne  MTADS  could  be 
straightforwardly  replicated  for  commercial  applications.  It  is  our  conclusion  that  the  current 
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MTADS  design  is  appropriate  for  direct  commercialization.  It  is  both  rugged  and  efficient.  The 
weak  components  were  replaced  during  the  shakedown  tests.  There  have  been  inquiries  from 
some  groups  about  potential  consulting  help  in  establishing  a  commercial  capability.  The 
impediments  are  the  substantial  capital  costs  involved  in  putting  a  commercial  system  together 
and  uncertainties  about  the  government’s  commitment  to  establishing  suitable  venues  for  its  use. 
If  a  request  for  proposal  (RFP)  were  issued  for  a  wide-area  UXO  search  (involving  several 
thousand  acres),  it  is  likely  that  there  would  be  multiple  responders  proposing  to  bring  in 
airborne  geophysics  (similar  to  the  Airborne  MTADS)  as  a  solution.  A  large  firm  would  likely 
want  25,000  acres  in  assured  airborne  UXO  survey  business  to  feel  they  could  recover  their 
investment  costs  and  potentially  make  a  profit.  Capitalizing  the  entire  system  cost  (see  Table  28) 
over  25,000  acres  would  add  only  ~  $25/acre  to  the  survey  costs. 

10.4  LESSONS  LEARNED 

Once  the  airborne  demonstration  began  on  the  IA,  it  took  place  almost  flawlessly.  There  were 
few  mistakes  or  failures  that  could  serve  as  learning  experiences.  This  is  in  contrast  to  our 
experience  during  the  shakedown  surveys.  There  were  three  shakedown  exercises  at  the 
Aberdeen  Proving  Ground  separated  by  1 -month  periods.  Each  was  dominated  by  equipment 
breakdowns,  malfunctions,  and  misadventures.  We  recovered  and  fixed  most  of  the  mistakes 
resulting  from  each  exercise  before  the  next  shakedown.  These  shakedown  exercises  were 
critical  to  the  success  of  the  final  IA  demonstration.  It  was  important  that  they  be  separated  by  at 
least  a  month  to  enable  us  to  evaluate  problems,  order  parts,  implement  fixes,  and  plan  for  the 
following  exercise. 

10.5  APPROACH  TO  REGULATORY  COMPLIANCE 

The  regulatory  issues  affecting  the  UXO  problem  are  most  frequently  associated  with  the  BRAC 
and  FUDS  processes  involving  the  transfer  of  DoD  property  to  other  government  agencies  or  to 
the  civilian  sector.  When  transfer  of  responsibility  to  other  government  agencies  or  to  the 
civilian  sector  takes  place,  the  DoD  lands  fall  under  the  compliance  requirements  of  the 
Superfund  statutes.  Section  2908  of  the  1993  Public  Law  103-160  then  requires  adherence  to 
CERCLA  provisions.  The  basic  issues  center  upon  the  assumption  of  liability  for  ordnance 
contamination  on  previously  DoD-controlled  sites.  These  regulatory  considerations  do  not  apply 
to  active  DoD  facilities. 

The  Airborne  MTADS  is  an  appropriate  technology  for  addressing  the  UXO  problem  in  areas 
where  the  terrain  cannot  be  traversed  on  foot,  that  are  dangerous  for  ground  activities,  and  that 
are  too  large  to  survey  economically  with  vehicular  systems.  These  demonstrations  provide  data 
to  demonstrate  a  statistical  probability  of  success  for  the  detection  and  characterization  of 
isolated  bombing  targets  or  impact  areas,  ordnance  burial  caches,  or  individual  ordnance, 
including  a  range  of  large  projectiles.  These  considerations  are  important  in  establishing  the 
value  of  this  approach  and  in  its  ultimate  acceptance  by  regulators  and  the  stakeholder 
community. 

Unless  it  is  necessary  to  clear  vegetation  to  allow  closer  access  to  the  ground,  it  is  possible  to 
conduct  effectively  noninvasive  airborne  surveys — without  coming  in  contact  with  the  ground  at 
any  point  on  the  range.  Hence,  there  is  no  chance  of  disturbing  cultural  or  religious  artifacts. 
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Even  within  active  ranges,  such  as  at  the  APG,  environmental  concerns  must  be  addressed 
because  soil  and  groundwater  contamination  by  energetic  residues  and  byproducts,  and  by  heavy 
metals  (As,  Bi,  Pb,  Sb,  U,  etc.)  associated  with  ordnance  components,  may  migrate  to 
underground  aquifers  and  routinely,  through  run-off,  reach  other  properties.  Specifically  at  the 
APG,  extensive  (on  base)  wetlands  are  used  by  migratory  birds  and  other  waterfowl;  and  marine 
estuaries  and  bays  beyond  the  APG  boundaries  (with  known  UXO  contamination)  are 
continually  harvested  for  fin  fish  and  shellfish  by  private  and  commercial  fishermen. 

Conducting  UXO  geophysical  surveys  in  shallow-water  wetlands  and  in  shallow  offshore  areas 
is  extremely  difficult,  expensive,  and  inefficient.  The  Airborne  MTADS  provides  a  technology 
appropriate  for  addressing  some  of  these  challenges. 

These  demonstrations  enabled  us  to  evaluate  the  extent  to  which  the  Airborne  MTADS  can  be 
effectively  applied  in  terrains  that  cannot  be  traversed  on  foot  and  in  areas  that  are  dangerous  for 
routine  ground  activities. 
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